Conditions of participation

• The challenge is open to everyone (above 18 years old) and there are no fees involved. 

• Members of the SKAO teams that have contributed to the simulation cannot compete.

• Participants from any country and any institution can compete. 

• Participation in the challenge implies acceptance of the competition rules by each participant. 

• Challenge registration will open through this website. There will be a deadline for registration, after which participation will not be possible. We reserve the right to close the challenge registration in advance if all available computational resources get allocated before the original deadline. 

Teams

• We advise participants to compete in teams. There is no minimum or maximum limit for the team size; however, allocation of computational facilities will be conditional on the teams being properly resourced. Every team will designate a team leader.

• Each team must join the competition using a dedicated registration form, that will be made available on this website, and accept the competition rules when joining the challenge.

• Participants are allowed to join an exisiting team during the competition.

• Any team or participant can request to withdraw from the competition.

Team leaders 

• The team leader is in charge of providing any material and information  that will be required.

• The team leader is also in charge of confirming people leaving/joining the team as well as requesting withdrawal of the whole team from the challenge.

• Team leaders can be changed upon request. 

Accessing and processing the data 

• Small versions of the SDC2 dataset, called the development dataset and the evaluation dataset, are available for download from this website for testing and validation purposes. The development dataset includes a truth catalogue. This truth catalogue is also a template demonstrating the format that your submitted results should follow in order to be considered.

• The complete (full) challenge dataset will be made available through a set of designated computing resources. Data analysis pipelines will need to be deployed at one of those facilities. 

• The resources provided will be diverse, but an effort will be made to ensure that the amount of computation offered is comparable and suitable to solve the challenge. 

• Details of the available resources, on the software environment and on options for deploying pipelines will be made available on the dedicated area of this website. 

• Each team will be assigned a particular computing resource. Whenever possible, we will follow the preference expressed by the teams in the allocations.  

• Preliminary allocation will follow the order of SDC2 registration requests and will be confirmed once the team performs a successful submission or results (see next section) for the evaluation dataset. This preliminary step will ensure that the team is capable of completing the challenge and that resources are used efficiently.

• The challenge duration will be 6 months, starting from when analysis will be able to commence at the designated processing facilities. The challenge deadline is 31st July 2021 at 23:59 UTC. 

• Participants should acknowledge the SDC2 computing facilities they used in any communication and publication regarding SDC2. 

• For more information on the SDC2 data, see the dedicated area of this website. 

Submissions and evaluation

• Submissions (consisting of catalogues of objects found in the SDC2 dataset) will be accepted throughout the duration of the challenge and will be scored through a scoring service. 

• In order to be successful, submissions should strictly comply with the requested format; the truth catalogue accompanying the development dataset serves as a template.

• The scoring procedure/service will be made available upon the start of the challenge. This will allow the self-assessment of a team's performance. The highest score obtained by each team will be updated on a public live leaderboard https://sdcss.skatelescope.org/leaderboard. 

• The Challenge winner will be awarded based on the leaderboard at the deadline. Submissions sent after the deadline will not be considered.

• In parallel to and independently from the SDC2 score, "reproducibility awards" will be awarded to teams whenever their pipeline meets a set of pre-defined criteria. More information will be available on this website in due course. 

Results/Publications

• The SDC2 leaderboard containing the final scores will be automatically published on this website once the challenge is closed. Leaderboards containing preliminary scores might be published during the challenge. 

• We reserve the right to publish details such as the name of the teams, their participants and affiliations, as well as their preliminary and final scores.

• After the completion of the challenge, we will explore the possibility of publishing results and methods in a scientific article, in collaboration with the participationg teams. 

Support

• Communication with the challenge organisers and support will happen through a dedicated community forum. Teams will be invited to the community forum upon registration to SDC2. 

• During the competition, the name of the teams and the names of the members of each team will be visible to the other challenge teams and participants through the forum platform.

• For IT support related to the computational resources, see the relevant section on the page of each facility. This support should essentially be used for access and initial setup. Teams are expected to be self-sufficient in deploying their pipelines.