How To Submit¶

For detailed information about the opening and closing dates of the submission phases, please visit the Important Dates page.
The challenge GitHub repository publishes the evaluation scripts employed for both tasks as well as example algorithms. 

Task 1: Fast Multi-class Segmentation¶

Debugging Phase¶

Teams are allowed to submit their algorithm through grand-challenge. The algorithms will be tested against four (4) cases taken from the training set (i.e., patients TODO, and TODO). The submissions can be tested up to 2 times per team per week. This phase is mainly designed to test the Docker images and the algorithms. We selected patients from the training case so that teams would be able to see logs for the output of their algorithm submission. As submission output, the grand-challenge evaluation code will produce the 95 Hausdorff distance (HD95) and Dice Score statistics per class. The total inference time will also be recorded.

The ranking is explained in the Ranking page.

Test Phase¶

The teams are required to submit their algorithm. The algorithm will be tested against fifty (50) hidden cases. The submission can be tested up to 3 times. The output of the evaluation script will be the same as the Debugging Phase, and the metrics used for the evaluations are described in the Ranking page.  Of the three submissions, the last received will be considered. The three best-performing teams will be awarded as challenge winners (see the Prize page for more details). Winners' algorithm must be publicly released and accompanied by a written report describing its main features.

Task 2: IAC Interactive Segmentation¶

Debugging Phase¶

Teams are allowed to submit their algorithm through grand-challenge. The algorithms will be tested against four (4) cases taken from the training set (i.e., patients TODO, and TODO). The submissions can be tested up to 2 times per team per week. This phase is mainly designed to test the Docker images and the algorithms. We selected patients from the training case so that teams would be able to see logs for the output of their algorithm submission. As submission output, the grand-challenge evaluation code will produce the Area Under the Click-to-DSC and Click-to-HD95 curves using the trapezoidal rule. In these curves, the x-axis represents the interaction step, while the y-axis reflects the DSC or HD95 achieved at that step. Additionally, the final DSC and HD95 metrics after all clicks are provided will be used to evaluate the model's overall performance. The evaluation script will also measure the execution time over all interaction steps and use it as one of the metrics to compute the final ranking, as interactive models must be both responsive and efficient to support a real-time human-in-the-loop annotation workflow.

The ranking schema is explained in the Ranking page.

Test Phase¶

The teams are required to submit their algorithm. The algorithm will be tested against fifty (50) hidden cases. The submission can be tested up to 3 times. The output of the evaluation script will be the same as the Debugging Phase, and the metrics used for the evaluations are described in the Ranking page.  Of the three submissions, the last received will be considered. The three best-performing teams will be awarded as challenge winners (see the Prize page for more details). Winners' algorithm must be publicly released and accompanied by a written report describing its main features.