If my diagnosis confidences are always either 0.0 or 1.0, will it be invalid? For examples
ISIC_0034336 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0
ISIC_0034376 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0
ISIC_0034345 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0
…
This should work for the ranking metric - have you tried uploading a validation set submission?
The submission system does not throw any error and it outputs overall score. Is there any disadvantages if submitted in this format?
The primary metric the ranking depends on should not be effected. It will have an effect on secondary metrics such as AUC calculations as there is only one cutoff.
Hi @rhinos,
Indeed, we’d strongly request that you submit your actual prediction probabilities within the full interval [0.0, 1.0]. This will allow us to compute other important metrics associated with the ROC curve of your algorithm.