We are sending an important announcement regarding a change in the threshold value we will use for the Personal Space Compliance (PSC) metric in the Social Navigation Challenge.
So far we have been using a PSC threshold of 0.5m. Through the past evaluation of participant’s policies, we discovered that this threshold is too small to meaningfully evaluate the politeness of the policies. Therefore, for the evaluation going forward, we will increase the PSC threshold to 1.5m.
Note that this threshold is defined as the distance between the centers of the pedestrian and the robot. Considering the radii of the pedestrian and the robot, using a threshold of 1.5m results in the desired distance of 1.2m between the closest points of the agents (border to border). We are setting this value modeling the desired social distance after the definition of interpersonal distances by Edward T. Hall (Proxemics - Wikipedia).
All the existing submissions have been re-evaluated using this new threshold and the leaderboard has been updated.
Thank you very much for your participation in iGibson Challenge 2021. Please let us know if you have any questions.
iGibson Challenge 2021 Organizers