Hi
I am not sure why but the vqa test-dev submission is giving me status as Failed. I have verified that the number of submissions is 107394 and is of the form of List[Dict] where each dictionary is of the form {"question_id": 123, "answer": "some string"}
Unfortunately, there is no message as to what went wrong. Can some direction be provided as to what needs to be fixed?
Hi,
If you scroll towards the right in the “My Submissions” page, you can see a column “Stderr File” which shows the error generated if the evaluation of your submitted file fails. Both of your submissions in Test-Dev phase have produced this error:
Results do not correspond to current VQA set. Either the results do not have predictions for all question ids in annotation file or there is atleast one question id that does not belong to the question ids in the annotation file. Please note that this year, you need to upload predictions on ALL test questions for test-dev evaluation unlike previous years when you needed to upload predictions on test-dev questions only.
Could you please ensure that your result file contains the answers to all the questions in the test set (not just test-dev)?
Please let us know if you have further questions. Happy to help.
Thank you. It worked after uploading the results on full test set. I didn’t note that there was a right scroll.