Although this was discussed in a previous post, I’m still not sure I totally get the metric that will be used for task 2 evaluation. The description says that it will use the Jaccard index over all images (since some ground truth labels are null). I wanted to be sure how the index is computed over multiple images.
My interpretation of this is that for each attribute for each image, you compute the intersection and the union, then sum intersections over all images/attributes, sum unions over all images/attributes and finally take the ratio between the two sums. Is that right?