Question about Evaluation Criteria


#1

From the Evaluation Criteria, I understood the definition of P (mIoU) and S (model-complexity), but what is the meaning of the min(1/S)? Is it min(1, S)?

From my understanding, the accuracy P higher is better and the model-complexity, S, lower is better, but from the second term of min(1/S), I cannot figure out what’s this.

Also, on the leaderboard, all of the total scores are seemed by measuring M = 50 * (mIoU + 1). The min(1/S) term is always 1.

Please give feedback about the measurement.


#2

Hi,

Please check https://evalai-forum.cloudcv.org/t/fyi-on-semantic-segmentation/180 for the details of evaluation metrics, we have fixed typo and explained the motivation of the evaluation metrics.