Status now updating from "submitted"


Hi @Winyi,

Thanks for reporting the issue and sorry for the inconvenience caused. We’ve resolved it and the submission is evaluated now.


Status is fixed at Submitted and evaluation isn’t starting.


I’m seeing the same thing, status never updates from submitted, usually it goes to running fairly quickly.


Hi @rishabh
I encountered the same problem, status got stuck at ‘Submitted’. Is it because the VQA challenge 2019 is over? Is it still possible to do evaluations here for other research purposes? Thanks!


Hi @qma, I assume it’s not because of that, I had one quickly evaluated yesterday. I also just had one run just now, though 2 I had submitted before it (and now 1 after) have not…


Hi @Saransh_Goyal, @cdr380, @qma,

The submissions are evaluated now. Please take a look at the results file. Thanks for reporting the issue.

Yes, the issue is fixed and it’ll run fairly quickly now.

No, it wasn’t because of it. The submissions are still open for the Dev and Test-Standard Phase throughout the year. Only the Test-Challenge phase submissions are closed.

Please feel free to report/ping us here if you face further problems. We’ll be happy to help.


Thanks @rishabh for the lightning reply. Indeed the results come out, you are amazing :smiley:


Hey @rishabh, there are problems with evaluation again today: one of my jobs has been ‘running’ for hours, and others jobs’ status got stuck at ‘Submitted’…Could you look into it? Thanks.


Hi @qma,

The evaluation is finished now for your submissions. Please check the results. Thanks for reporting it.


Hi @rishabh, my recent two submissions have been stuck at “Submitted” for more than an hour. I was wondering if you could take a look. Thanks!


Hi @playerkk, The submissions are evaluated now. Please check the results.



The issue appears again, can you help me to solve it? Thanks


Hi @Zhicheng_Huang,
Thanks for reporting it. It is fixed now.


Thanks for your help, but now the issue appears again.


Hi @Zhicheng_Huang,

Thanks for reporting it again, the issue is fixed now.


@rishabh can you help me solve this issue, it appears again.Thanks


@rishabh can you help solve this issue, it appears again. Thanks.


Confirm the issue is still there.


Hi @zengzhaoyang and @playerkk, Thanks for reporting. It is fixed now. The submissions will be evaluated in few mins.


Thank you very much for maintaining the evaluation server!