Call for Papers

Overview

The 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP 2021) invites the submission of long and short papers on substantial, original, and unpublished research in empirical methods for Natural Language Processing. As in recent years, some of the presentations at the conference will be for papers accepted by the Transactions of the ACL (TACL) and Computational Linguistics (CL) journals.

Important Dates

  • Anonymity period begins: April 17, 2021
  • Abstract submission deadline (long & short papers): May 10, 2021
  • Full paper submission deadline (long & short papers): May 17, 2021
  • Author response period: July 11-17, 2021
  • Notification of acceptance (long & short papers): August 25, 2021
  • Camera-ready papers due (long & short papers): September 9, 2021
  • Conference: November 7-9, 2021
  • Workshops & Tutorials: November 10-11, 2021

All deadlines are 11.59 pm UTC -12h ("anywhere on Earth").

Mandatory abstract submission

The paper title, author names, contact details, and a brief abstract must be submitted electronically through the EMNLP 2021 paper submission site by the abstract submission deadline. It will be possible to make minor edits to the title and abstract until the full paper submission deadline but you cannot change authors and subject areas. Submissions with “placeholder” abstracts will be removed without consideration; there is no need to withdraw the submission or notify us. Important: if you miss the abstract submission deadline, then you cannot submit the full paper.

Submissions

EMNLP 2021 has the goal of a broad technical program. Relevant topics for the conference include, but are not limited to, the following areas (in alphabetical order):

  • Computational Social Science and Cultural Analytic
  • Dialogue and Interactive Systems
  • Discourse and Pragmatics
  • Efficient Methods for NLP
  • Ethics and NLP
  • Generation
  • Information Extraction
  • Information Retrieval and Text Mining
  • Interpretability and Analysis of Models for NLP
  • Linguistic Theories, Cognitive Modeling and Psycholinguistics
  • Machine Learning for NLP
  • Machine Translation and Multilinguality
  • NLP Applications
  • Phonology, Morphology and Word Segmentation
  • Question Answering
  • Resources and Evaluation
  • Semantics: Lexical, Sentence level, Textual Inference and Other areas
  • Sentiment Analysis, Stylistic Analysis, and Argument Mining
  • Speech, Vision, Robotics, Multimodal Grounding
  • Summarization
  • Syntax: Tagging, Chunking and Parsing

Paper Submission Information

Long Papers

Long paper submissions must describe substantial, original, completed and unpublished work. Wherever appropriate, concrete evaluation and analysis should be included. Review forms will be made available prior to the deadlines. Long papers may consist of up to 8 pages of content, plus unlimited pages for references; final versions of long papers will be given one additional page of content (up to 9 pages) so that reviewers’ comments can be taken into account. Long papers will be presented orally or as posters as determined by the program committee. The decisions as to which papers will be presented orally and which as poster presentations will be based on the nature rather than the quality of the work. There will be no distinction in the proceedings between long papers presented orally and as posters.

Short Papers

Short paper submissions must describe original and unpublished work. Please note that a short paper is not a shortened long paper. Instead short papers should have a point that can be made in a few pages. Some kinds of short papers are:

  • A small, focused contribution
  • A negative result
  • An opinion piece
  • An interesting application nugget

Short papers may consist of up to 4 pages of content, plus unlimited references. Upon acceptance, short papers will be given 5 content pages in the proceedings. Authors are encouraged to use this additional page to address reviewers’ comments in their final versions.

Short papers will be presented orally or as posters as determined by the program committee. While short papers will be distinguished from long papers in the proceedings, there will be no distinction in the proceedings between short papers presented orally and as posters.

Authorship

The author list for submissions should include all (and only) individuals who made substantial contributions to the work presented. Each author listed on a submission to EMNLP 2021 will be notified of submissions, revisions and the final decision. No changes to the order or composition of authorship may be made to submissions to EMNLP 2021 after the abstract submission deadline.

Citation and Comparison

You are expected to cite all refereed publications relevant to your submission, but you may be excused for not knowing about all unpublished work (especially work that has been recently posted and/or is not widely cited).

In cases where a preprint has been superseded by a refereed publication, the refereed publication should be cited instead of the preprint version. Papers (whether refereed or not) appearing less than 3 months before the submission deadline are considered contemporaneous to your submission, and you are therefore not obliged to make detailed comparisons that require additional experimentation and/or in-depth analysis.

For more information, see the ACL Policies for Submission, Review, and Citation

Multiple Submission Policy

EMNLP 2021 will not consider any paper that is under review in a journal or another conference at the time of submission, and submitted papers must not be submitted elsewhere during the EMNLP 2021 review period. This policy covers all refereed and archival conferences and workshops (e.g., NeurIPS, ACL workshops). In addition, we will not consider any paper that overlaps significantly in content or results with papers that will be (or have been) published elsewhere. Authors submitting more than one paper to EMNLP 2021 must ensure that their submissions do not overlap significantly (>25%) with each other in content or results.

Ethics Policy

Authors are required to honour the ethical code set out in the ACL Code of Ethics. The consideration of the ethical impact of our research, use of data, and potential applications of our work has always been an important consideration, and as artificial intelligence is becoming more mainstream, these issues are increasingly pertinent. We ask that all authors read the code, and ensure that their work is conformant to this code. Where a paper may raise ethical issues, we ask that you include in the paper an explicit discussion of these issues, which will be taken into account in the review process. We reserve the right to reject papers on ethical grounds, where the authors are judged to have operated counter to the code of ethics, or have inadequately addressed legitimate ethical concerns with their work.

Authors will be allowed extra space after the 8th page (4th for short papers) for a broader impact statement or other discussion of ethics. The EMNLP review form will include a section addressing these issues and papers flagged for ethical concerns by reviewers or ACs will be further reviewed by an ethics committee. Note that an ethical considerations section is not required, but papers working with sensitive data or on sensitive tasks that do not discuss these issues will not be accepted. Conversely, the mere inclusion of an ethical considerations section does not guarantee acceptance. In addition to acceptance or rejection, papers may receive a conditional acceptance recommendation. Camera-ready versions of papers designated as conditional accept will be re-reviewed by the ethics committee to determine whether the concerns have been adequately addressed. Please read the ethics FAQ for more guidance on some problems to look out for and key concerns to consider relative to the code of ethics.

Paper Submission and Templates

Submission is electronic, using the Softconf START conference management system. Both long and short papers must follow the EMNLP 2021 two-column format, using the supplied official style files. Please do not modify these style files, nor should you use templates designed for other conferences. Submissions that do not conform to the required styles, including paper size, margin width, and font size restrictions, will be rejected without review.

Optional Supplementary Materials: Appendices, Software and Data

Each EMNLP 2021 submission can be accompanied by one PDF appendix for the paper, one PDF for prior reviews and author response, one .tgz or .zip archive containing software, and one.tgz or .zip archive containing data. EMNLP 2021 encourages the submission of these supplementary materials to improve the reproducibility of results, and to enable authors to provide additional information that does not fit in the paper. For example, anonymised related work (see above), preprocessing decisions, model parameters, feature templates, lengthy proofs or derivations, pseudocode, sample system inputs/outputs, and other details that are necessary for the exact replication of the work described in the paper can be put into the appendix. However, the paper submissions need to remain fully self-contained, as these supplementary materials are completely optional, and reviewers are not even asked to review or download them. If the pseudo-code or derivations or model specifications are an important part of the contribution, or if they are important for the reviewers to assess the technical correctness of the work, they should be a part of the main paper, and not appear in the appendix. Supplementary materials need to be fully anonymized to preserve the double-blind reviewing policy.

Anonymity Period

The following rules and guidelines are meant to protect the integrity of double-blind review and ensure that submissions are reviewed fairly. The rules make reference to the anonymity period, which runs from 1 month before the submission deadline (starting April 17th, 2021) up to the date when your paper is accepted or rejected (August 25th, 2021). Papers that are withdrawn during this period will no longer be subject to these rules.

  • You may not make a non-anonymized version of your paper available online to the general community (for example, via a preprint server) during the anonymity period. Versions of the paper include papers having essentially the same scientific content but possibly differing in minor details (including title and structure) and/or in length.
  • If you have posted a non-anonymized version of your paper online before the start of the anonymity period, you may submit an anonymized version to the conference. The submitted version must not refer to the non-anonymized version, and you must inform the programme chairs that a non-anonymized version exists.
  • You may not update the non-anonymized version during the anonymity period, and we ask you not to advertise it on social media or take other actions that would further compromise double-blind reviewing during the anonymity period.
  • You may make an anonymized version of your paper available (for example, on OpenReview), even during the anonymity period.
  • Note that, while you are not prohibited from making a non-anonymous version available online before the start of the anonymity period, this does make double-blind reviewing more difficult to maintain, and we therefore encourage you to wait until the end of the anonymity period. Alternatively, you may consider submitting your work to the Computational Linguistics journal, which does not require anonymization and has a track for “short” (i.e., conference-length) papers.
  • For arXiv submissions, April 17 11:59pm UTC-12h (anywhere on earth) is the latest time the paper can be uploaded.

Instructions For Double-Blind Review

As reviewing will be double blind, papers must not include authors’ names and affiliations. Furthermore, self-references or links (such as github) that reveal the author’s identity, e.g., “We previously showed (Smith, 1991) …” must be avoided. Instead, use citations such as “Smith previously showed (Smith, 1991) …” Papers that do not conform to these requirements will be rejected without review. Papers should not refer, for further detail, to documents that are not available to the reviewers. For example, do not omit or redact important citation information to preserve anonymity. Instead, use third person or named reference to this work, as described above (“Smith showed” rather than “we showed”). If important citations are not available to reviewers (e.g., awaiting publication), these paper/s should be anonymised and included in the appendix. They can then be referenced from the submission without compromising anonymity. Papers may be accompanied by a resource (software and/or data) described in the paper, but these resources should also be anonymized.

Sticky Reviews (optional)

Authors resubmitting a paper that has been rejected from another venue are invited to submit alongside their paper the previous version of the paper, the reviews and an author response, following the procedure introduced at EMNLP last year. This is optional. It is designed to mimic the revise-and-resubmit procedure underlying journals like TACL. We expect that the fact that a paper was rejected from another venue will not necessarily affect the paper’s decision in a negative way, but is likely to be beneficial to authors who believe they have addressed the problems identified, and can argue strongly for how the paper has been improved. The prior reviews will not be seen by reviewers, but be used as part of the EMNLP 2021 decision process, primarily by area chairs and program chairs in review quality control, resolving disagreements between reviewers, and in deciding borderline papers.

Reproducibility Criteria

During the submission process, authors will be asked to answer the questions from the Reproducibility Checklist. The checklist is intended as a reminder to help the authors improve reproducibility of their papers. The papers are not required to meet all reproducibility criteria listed. However, the answers will be made available to the reviewers. Reviewers will be asked to assess the reproducibility of the work as part of their reviews.

The following is a preliminary checklist we plan to use. For all reported experimental results:

  • A clear description of the mathematical setting, algorithm, and/or model.
  • Submission of a zip file containing source code, with specification of all dependencies, including external libraries, or a link to such resources (while still anonymized)
  • Description of computing infrastructure used
  • The average runtime for each model or algorithm (e.g., training, inference, etc.), or estimated energy cost
  • Number of parameters in each model
  • Corresponding validation performance for each reported test result
  • Explanation of evaluation metrics used, with links to code

For all experiments with hyperparameter search:

  • The exact number of training and evaluation runs
  • Bounds for each hyperparameter
  • Hyperparameter configurations for best-performing models
  • Number of hyperparameter search trials
  • The method of choosing hyperparameter values (e.g., uniform sampling, manual tuning, etc.) and the criterion used to select among them (e.g., accuracy)
  • Summary statistics of the results (e.g., mean, variance, error bars, etc.)

For all datasets used:

  • Relevant details such as languages, and number of examples and label distributions
  • Details of train/validation/test splits
  • Explanation of any data that were excluded, and all pre-processing steps
  • A zip file containing data or link to a downloadable version of the data
  • For new data collected, a complete description of the data collection process, such as instructions to annotators and methods for quality control.

This list is based on Dodge et al, 2019 and Joelle Pineau's reproducibility checklist.

Presentation Requirement

All accepted papers must be presented at the conference to appear in the proceedings.
Authors of papers accepted for presentation at EMNLP 2021 must notify the program chairs by the camera-ready deadline if they wish to withdraw the paper.

At least one author of each accepted paper must register for EMNLP 2021 by the early registration deadline.

ACL Rolling Review

We are working with ACL Rolling Review towards running a pilot with EMNLP 2021. The plan is for a small proportion of our papers to come from ARR.

ARR is a new review system for *ACL conferences, where reviewing and acceptance of papers to publication venues is done in a two-step process: (1) centralized rolling review and (2) submission to a publication venue. The purpose of ACL Rolling Review is to improve efficiency and turnaround of reviewing in *ACL conferences while keeping the diversity (geographic and otherwise) and editorial freedom.

Cut-off dates for publication and presentation at EMNLP 2021

EMNLP will only consider papers and reviews from ARR if: (1) the papers are submitted to ARR by May 15, 2021 (i.e., the first deadline for ARR), and (2) the reviews are ready by July 29, 2021. Given that this is the first ARR pilot and is running in parallel with the traditional reviewing pipeline, if EMNLP receives a large number of submissions from ARR, we may cap the number of papers that will be evaluated by EMNLP (e.g., 100 submissions).


More information can be found in the Committee blog. If you have questions that are not answered there, please email the program co-chairs at emnlp2021pcs@gmail.com.