Task 2 Groundtruth: empty globules masks? Small amount of annotations?

ISIC 2018 Challenge
1

Hi all,

I downloaded version 2 of Task 2 ground truth. After checking the attribute masks (milia like, streaks, etc.) I found that the globules masks (ISIC_XXXXXXX_attribute_globules.png) are completely black images. Therefore, I understand that there are no globules annotations at all in the database. Am I missing something or is this the case?

Additionally, I found this prevalence of images with at least 25 pixels annotated with lesions:
Pigment network: ~58% of the total amount of images
Negative network: ~7% of the total amount of images
Streaks: ~1,5% of the total amount of images
Milia like cyst: ~26% of the total amount of images
Globules: 0% of the total amount of images

Under these conditions, does it make sense to train a system for Streak or Negative network segmentation?

Thanks

2

Hello @melanoma

We will investigate this immediately let you know what we find.

3

Hi Brian,

Do you have any updates on this issue? Same situation here. The sample scarcity makes it difficult to make correct prediction.

Thanks!

4

Hello @melanoma and @cx.lavinia ,

Thank you for your patience. I wanted to fully investigate the issue and verify our solution, but we now have an official update:

We recently became aware of an issue where certain attributes were severely under-represented in the ISIC Challenge’s Task 2: Lesion Attribute Detection training ground truth. Accordingly, we’ve re-generated and re-released the training ground truth, after thoroughly re-querying our database for all available ground truth data.

The updated training ground truth bundle may be downloaded from: https://challenge.kitware.com/#phase/5abcbb6256357d0139260e5f , by the filename ISIC2018_Task2_Training_GroundTruth_v3.zip . It is necessary that all Task 2 participants download and use this updated bundle for training algorithms. Please discard all earlier versions of the Task 2 training ground truth bundle. However, the training input data bundle (containing JPEG lesion images) has not changed and does not need to be re-downloaded.

For reference, the image-wise counts of positive attribute instances in this updated data are:
globules: 603
milia_like_cyst: 682
negative_network: 190
pigment_network: 1523
streaks: 100

We certainly regret any inconveniences that this may have caused. Please let me know here if you have any further questions or issues.

5

Thank you very much Brian,

I will try with this new batch. May I ask another related question? I noticed that some Task 2 annotations have been made outside the delineated (annotated) mole region for Task 1. Is this correct?

Thanks in advance

6

Hi @melanoma ,

Rarely, dermoscopic attributes may be present outside of the image’s primary lesion (perhaps within a secondary lesion within the image), but this is not common.

7

Hello @Brian

i found some of the cases like 4, 23, 40, 99, 101, 110 … in latest release of GroundTruth-v3 for Task 2 all the mask images are blank. are they really blank or some correction is to be done?

Thanks

8

There are lesions with none of the 5 dermoscopic attributes. These serve as negative examples for training.

9

Thank you