Task 3 Submission requires ordering by 'image' column

ISIC 2018 Challenge
1

Hello,

I have an issue with the submission, my algorithm does not sort rows by the first - ‘image’ column by default.
When the output IS sorted by the first column, my validation score becomes almost 5x times bigger than for the as-is output.

I double checked that the results are equivalent (except for the sort order).

2

Andrey— Thanks for catching this. I’ll make sure the team notes that— we clearly assumed the output of the images was in a certain order, but if we don’t specify that, of course it will result in squirrely results like you noticed… Really sorry about that; will let you know and/or send an update once we update the instructions and/or FIX the issue…

1 Like
3

Thanks for the fast response, David! I changed the algorithm to return sorted data and scores improved, just wanted to make this potential issue visible to other participants.

4

Hi @andrey.i.sorokin,

The scoring has been updated to handle the unsorted case. Thanks for raising this issue.

1 Like