How are the leaderboards ranked?

Argoverse Motion Forecasting Competition
1

image
image2308×1014 166 KB

How are the leaderboards ranked? The slider button on the top right says “sort by best” - how is “best” defined? There are four metrics for three cases and it looks like ranks could differ depending on what combination of cases/metrics you look at.

2

Hi @aaronhuang,

The leaderboard is sorted according to the best submission by a team on a particular metric. Please see the latest leaderboard and ignore this screenshot.

3

Thank you for your response. It appears that the submissions are ranked by the K=6 metrics. However, the leaderboard ranks submissions in decreasing order of minADE/minFDE.

If I understand the metrics correctly, we wish to achieve a lower minADE/minFDE since that implies trajectories that deviate less from the ground truth trajectory right? Similarly, we wish to have a lower miss rate (MR), but the rankings appear to be reversed.

4

Hey Aaron,
The leaderboard should be properly ranked now. Let us know if you have any other issue.

Best,
Patsorn