Submitted models is not tested

Hi, I have a quick question about the testing.

My last submission is on May 18, 2020 11:54:59 PM and the status is still ‘Submitted’ and no error message returned. Therefore I can not submit new models. I was wondering if there was something wrong with my submission.

The submitted file link is

{"submitted_image_uri": ""}

Any suggestion will be helpful. Thanks very much.

Sorry for the delay. The submission has been processed. One issue I ran into with your submission was the version of werkzeug was not high enough (this was previously not defined). I have added the min version required to the robothor-challenge requirements.txt in the repo. Before your next submission please pull from master.


Thanks for your help.

Sorry to reopen this topic. I have submitted a model for the “Test Challenge Sim RGB” phase on Jun 11, 2020 9:38:22 PM. The status is still Submitted. Could you please help me to confirm that everything is going well with my submission? Thanks very much.

The submitted file link is

{"submitted_image_uri": ""}

Your best performing public submission has been submitted on your behalf to the challenge phase. Evaluation is still running (it was started ~2 days ago) - it has finished about 60% of the episodes. Do you know if your model uses the GPU it appears to be idle on our evaluation server?

thank you,

Thanks for your help. Our model has two parts – frame feature extraction and action prediction. The first part is running on Tensorflow using CPU which I think leads to the low inference speed. (The second part is running on PyTorch using GPU.) I think I underestimated the consequences of slow reasoning. Sorry for the trouble.

Hi,would you please help me to check if the mode testing process finished? Thanks very much.

Your model is 96% complete. It should complete within the next 12 hours.