I tried to submit my result file for nuScenes prediction challenge though URL. But the my submissions either failed without any trace info(stderr output seemed to be empty) or got a bad address error (error report is shown below). Does anyone meet the same issue? FYI, I used dropbox to create my url link and it was tested by wget command.
Traceback (most recent call last):
File "/code/scripts/workers/submission_worker.py", line 452, in run_submission
File "/tmp/tmpwzsasqbb/compute/challenge_data/challenge_591/main.py", line 73, in evaluate
predictions = json.load(open(result_path, "r"))
File "/usr/local/lib/python3.6/json/__init__.py", line 296, in load
OSError: [Errno 14] Bad address
Hi @Kay1794, I checked your submissions and I saw that the evaluation server is failing while reading the file you are uploading. Can you please confirm if the file is zipped correctly?
Here’s the error returned by evaluation server.
I checked the submission format. Each submission should have just the following keys “instance”, “sample”, “prediction”, “probabilities”. Also I cross checked with other submissions the number of test samples are 9041 in successful submissions but your submissions have 316727 prediction samples which is causing evaluation server to fail. Can you please verify whether you are creating submission file correctly for the phase you are submitting to.
Hi @ram81, thank you for your help. I used the official code to generate the result file. My understanding is that 9041 is the sample number instead of the instance number. Anyway, thank you so much for cross checking with other submission files. I’ll double check the nuScenes webpage in case I have misunderstood something. Thank you for your time again!
Hi @ram81, thank you for your help before, so I resubmitted the newly generated result file which has 9041 samples and had waited for more than 24 hrs. However, currently the status is still submitted. I was wondering if it is normal and what the typical time it will take to evaluate a result.
Hi @Kay1794, I fixed the issue with the evaluation worker but your submission is still failing. It seems like your new submission is same size as your previous submissions which is why our workers are failing to evaluate your submission. Total samples are still 316727. Can you please confirm if you uploaded the correct submission file?
Hi @ram81, so sorry…put an incorrect url. I reuploaded the file(double-checked sample number) and this time I uploaded the file directly instead of using a url link as I did before. The status took around 30 secs turning ‘running’ to ‘failed’ without any error report. Then I tried upload again, now it was stuck at “submitted” status again…