Hello,

Could you please tell me how the ISIC score the prediction for task 3 if it has two maximal positions such as [0, 0, 0, 0.5, 0, 0.5]? Thanks!

Hi @tat-dat,

The short answer is that such a row is always scored as false / incorrect, so youâ€™d be much better to break any ties (add or subtract a very small value from one of the tied probabilities) before submitting (even if you had to do so randomly).

We have already considered that this situation may happen. In such a case of more than one maximal value, we consider that the detector is â€śundecidedâ€ť (which we consider to not be a diagnostically useful category). Internally, our scoring confusion matrix actually counts this image as having a diagnosis of â€śundecidedâ€ť too; of course, since the ground truth contains no â€śundecidedâ€ť samples, such a diagnosis is always wrong.

I hope this helps.

1 Like

Thank you very much!