In partnership with the Software Sustainability Institute (SSI), we will be awarding a set of reproducibility awards to all teams whose pipelines demonstrate best practice in the provision of reproducible results and reusable methods. An essential part of the scientific method, reproducibility leads to better, more efficient science. Reusability generalises this principle to create software that can be adapted by others, allowing previous work to be built upon for the future: a key feature of open science.

Reproducibility awards will run in parallel and independently from the SDC2 score, and there is no cap on the number of teams to whom the awards can be given. All pipelines will be evaluated at the close of the challenge using the criteria set out in a downloadable checklist:

We encourage each team to discuss early on in the challenge the overall architecture and design of their software pipeline, in order to identify and agree upon which practices will be put in place during pipeline development. The SSI provide a fantastic collection of guides to software best practice; the top five don'ts of software development is a great place to start. Several more guides are linked in the checklist, alongside the relevant award criteria.

This checklist can be used for self-assessment by teams during the challenge. Colour coding is used to indicate three levels of award: bronze, silver and gold.

All parts of the software pipeline that have been developed by each team will be evaluated. This includes packages that the team have written and code that interacts with third party packages, but does not include any third party packages themselves.