I know this is very last minute, but would it be logistically feasible to have a side submission — not counting for the prizes, and not counting in the allowed quota of submissions for the prizes — for an informal “melanoma vs all” subtask?
We are observing very noisy results for the official task (due to the unbalanced tasks), and feel it would be extremely informative for our ongoing research to have this secondary informal task where the models are optimized just for melanoma. You could add or not the unknown class to this.
We will have many metrics reported – melanoma vs. all will be one of them. I recommend you save this optimization for the live challenge (after the official prize associated challenge).
The official challenge metric is still balanced accuracy (with the addition of the out-of-distribution class).
Dear Eduardo,
I found in here there are many problems to solve. at first glance, all data need a shallow pre processing such hair remove or remove shadow. then classes aren’t balance. F1 is better than accuracy in classification metric.