In the interest of maximizing fairness and the meritocratic nature of the evaluation process, and to minimize the impact of possible implicit biases, ECRTS implements a double-blind peer reviewing process.
Concretely, this means that authors will submit blinded manuscripts (that do not reveal author identity or affiliation) and reviewers will not be made aware of author identities until after acceptance/rejection decisions have been made. Papers not selected by the program committee for acceptance will not be de-blinded.
We ask that all authors and program committee members make an honest effort to comply with both the letter and the spirit of the following double-blind submission rules. Failure to do so would be unethical and may be considered research misconduct.
Rationale
Arguably no double-blind peer reviewing process can guarantee perfect anonymity (given community size, well-known topic preferences, writing styles, resubmissions, availability of preprints, etc.). It is also not difficult to imagine ways in which a double-blind peer reviewing process might be partially subverted by scheming individuals.
Nonetheless, even an imperfect double-blind review process is much better than none when it comes to improving fairness, lessening the potential impact of implicit biases, etc.
At the same time, we recognize that the double-blind process can require a significant amount of extra effort by the authors, especially when it comes to the discussion of their own prior publications and artifacts. Therefore, we are committed to implementing a double-blind peer reviewing process to ensure proper author anonymity without unduly burdening the authors and preventing papers from being submitted.
Double-blind submission rules
Generally: when in doubt, ask the Program Chair for clarification or guidance prior to submission.
- Submitted manuscripts must not reveal (or hint at) author identities, affiliations, funding sources, or major project names or initiatives. This rule pertains to the paper content and PDF metadata as well as any data sets, research artifacts, etc. The submitted manuscript should not contain any acknowledgments; they can be added to the camera-ready version if the paper is accepted.
- Regarding related prior work by the authors, the goal is to strike a balance between mantaining anonymity and providing the reviewers with sufficient information to properly evaluate the submission. The following rules apply:
- Prior work by the authors that has been peer reviewed and published must be discussed and cited just like any other related work by adopting a third-party point of view (i.e., using author names and third-person pronouns rather than โweโ/โourโ). References must not be omitted or blinded, as this hinders the peer-reviewing process.
- Related work by the authors that is either under peer review submission, or has been accepted but not published yet (and thus it is not publicly available), must be provided at submission time in a suitably blinded form as part of the supplemental materials.
- As an exception to Rule 1, own prior workshop and work-in-progress publications that present a preliminary version of the same work as the submitted manuscript (in part or in full), are exempted from this rule to not disadvantage authors who choose to solicit feedback on ongoing work. Such preliminary versions should be disclosed to the Program Chair at the time of submission through the submission form, but must not be discussed in the submitted manuscript itself. The same rule applies to other non-peer reviewed materials that present a preliminary version of the submitted work, such as technical reports or Master/PhD theses. For prior workshop and work-in-progress papers which appeared in formal proceedings, the Program Chair reserves the right to ask the authors to provide a suitably blinded version of the paper after submission.
- If the submitted manuscript depends on an existing software or hardware project, dataset, or other artifact that has been publicly announced (for example, through a previous publication), then it must be possible to plausibly justify how the authors acquired the artifact without inferring their identities or affiliations. If the artifact is open source, then no explanation is needed. If the artifact was announced in a prior work by the authors and it is not publicly available, then the manuscript should include a note thanking the authors of the prior work for making the artifact available to the authors of the present paper.
- As an exception to the previous rule, in rare cases it might be impossible to plausibly justify how the authors acquired the artifact without revealing their affiliations (for example, the manuscript might depend on pre-production hardware that could not be reasonably made available outside of a specific company). In this case, all references to the artifact should be blinded by substituting its name with a placeholder. If this is not possible or it would lead to removing from the paper essential information, the authors should contact the Program Chair by email to receive case-specific instructions.
- The authors should avoid reusing text and figures from previous work published in formal proceedings as much as possible, also in the interest of avoiding any impression of (self-)plagiarism. When unavoidable, the source material should be cited (e.g., โ(figure adapted from [X])โ).
- If a submitted manuscript refers to supplemental materials of a textual of visual nature (e.g., an extended online appendix or supporting tech report, demo videos, etc.), then any such supplemental materials must be provided in suitably blinded form. The authors can upload the related files directly to the conference management system, or to a hosting site such as Zenodo which allows artifacts to be listed and referenced anonymously. If neither solution is possible (for example, because of the size of the file), then the authors should contact the Program Chair by email. In either case, supplemental materials must be made available at the time of submission.
- If the supplemental materials consist of software or hardware code in a repository, then as an exception to the previous rule, the authors are allowed to blind the reference from the submitted manuscript and do not need to submit the code along with the paper. However, the reference must be disclosed to the Program Chair through the submission form. This is in recognition of the fact that anonymizing a code repository could be too onerous for the authors.
- Authors are allowed to publish a preprint of their manuscript to public websites (e.g., preprint servers such as arXiv, personal, project, or institution websites, etc.); we recognize that this is often required to meet sponsor, institutional and/or project requirements. Similarly, authors are allowed to upload software or hardware code related to the manuscript to a public repository (e.g., GitHub or GitLab). Authors are also allowed to discuss their work directly with colleagues in an informal setting; we do not intend to hamper collaborations. However, until notification of acceptance, authors must not advertise their work (e.g., on social media platforms, mailing lists, etc.).
- Failure to comply with these rules can result in rejection without further review.
Double-blind rules for reviewers
Reviewers are explicitly prohibited from deliberately trying to de-anonymize a manuscript. For example, they should not try to search the web nor public archives / repositories (e.g., arXiv, GitHut, GitLab, etc.) based on the title of the paper or related keywords, nor should they search the personal or institutional websites of colleagues they might suspect to be authors.
In some cases, even if the authors follow the double-blind rules with their best intentions, it might be possible for a reviewer to infer their identities or affiliations. For example:
- An author might have privately discussed the project with the reviewer;
- The reviewer might have reviewed a previous version of the manuscript submitted to a non double-blind venue;
- The reviewer might have found a preprint of the manuscript online (without explicitly trying to de-anonymize it after seeing the list of submitted papers).
In either case, the reviewer is still allowed to review the paper and should not declare a conflict of interest. The reviewer should not reveal the identities nor affiliations of the authors, and should make a conscious effort to avoid biases when evaluating the submission.
If, while reviewing the paper, a reviewer believes that the authors can be identified due to a violation of the double-blind rule, then they should immediately contact the Program Chair and not reveal the authorsโ identities to the rest of the committee while a decision is taken.
In case a PC member has reviewed a previous version of the manuscript submitted to another conference, they should avoid copying their previous review, even in part, as this might allow the authors to deduce the identity of the reviewer by cross-checking the PC member lists.