Task 1, validation, Jaccard Index

ISIC 2018 Challenge
1

Hi,

For Task 1, I submitted my validation result and got the average score. The problem is that when I check the values of Jaccard Index for each image separately, sometimes it’s exactly 0.00e+0 but visually the segmentation of lesion seems to be correct. Is it normal?

Thank you,
Best

2

Hi,
I got the same Jaccard Index on below validation images;
|ISIC_0012627|0.00e+0|
|ISIC_0015492|0.00e+0|
|ISIC_0017702|0.00e+0|
|ISIC_0019794|0.00e+0|
|ISIC_0020999|0.00e+0|
|ISIC_0021037|0.00e+0|
|ISIC_0021904|0.00e+0|
|ISIC_0021914|0.00e+0|
|ISIC_0023508|0.00e+0|
|ISIC_0036073|0.00e+0|
|ISIC_0036085|0.00e+0|
|ISIC_0036098|0.00e+0|
|ISIC_0036247|0.00e+0|

What is the problem?

3

As far as I see the lesion-wise numbers on the evaluation platform already reflect the thresholded Jaccard metric used in this years challenge, i.e. if Jaccard for a single lesion <0.65 it displays and counts as 0.

4

Thanks Philipp for your info.