Submission failed: bad address

1

Hi,

I tried to submit my result file for nuScenes prediction challenge though URL. But the my submissions either failed without any trace info(stderr output seemed to be empty) or got a bad address error (error report is shown below). Does anyone meet the same issue? FYI, I used dropbox to create my url link and it was tested by wget command.

Traceback (most recent call last):
  File "/code/scripts/workers/submission_worker.py", line 452, in run_submission
    submission_metadata=submission_serializer.data,
  File "/tmp/tmpwzsasqbb/compute/challenge_data/challenge_591/main.py", line 73, in evaluate
    predictions = json.load(open(result_path, "r"))
  File "/usr/local/lib/python3.6/json/__init__.py", line 296, in load
    return loads(fp.read(),
OSError: [Errno 14] Bad address
2

Hi @Kay1794,

Sorry for the inconvenience caused. We had an issue with the worker yesterday and hence the submission got failed. Please try the submission again and it should work now.

3

Thank you very much for the reply! So I just resubmitted and it has shown submitted status for more than two hours. Is that normal?

4

Hi @Kay1794, I just re-ran your submission. It should be done evaluating in some time.

5

Hi @ram81, thank you for your response but I still see the status of my submissions are failed/cancelled…

6

Hi @Kay1794, I checked your submissions and I saw that the evaluation server is failing while reading the file you are uploading. Can you please confirm if the file is zipped correctly?
Here’s the error returned by evaluation server.

7

Hi @ram81, I download the submitted file and unzipped it successfully. It contains one json file named result.json. I was wondering if it is the correct format for the submission.

8

Hi @Kay1794,

I checked the submission format. Each submission should have just the following keys “instance”, “sample”, “prediction”, “probabilities”. Also I cross checked with other submissions the number of test samples are 9041 in successful submissions but your submissions have 316727 prediction samples which is causing evaluation server to fail. Can you please verify whether you are creating submission file correctly for the phase you are submitting to.

9

Hi @ram81, thank you for your help. I used the official code to generate the result file. My understanding is that 9041 is the sample number instead of the instance number. Anyway, thank you so much for cross checking with other submission files. I’ll double check the nuScenes webpage in case I have misunderstood something. Thank you for your time again!

10

Hi @ram81, thank you for your help before, so I resubmitted the newly generated result file which has 9041 samples and had waited for more than 24 hrs. However, currently the status is still submitted. I was wondering if it is normal and what the typical time it will take to evaluate a result.

11

Hi @Kay1794, I fixed the issue with the evaluation worker but your submission is still failing. It seems like your new submission is same size as your previous submissions which is why our workers are failing to evaluate your submission. Total samples are still 316727. Can you please confirm if you uploaded the correct submission file?

12

Hi @ram81, so sorry…put an incorrect url. I reuploaded the file(double-checked sample number) and this time I uploaded the file directly instead of using a url link as I did before. The status took around 30 secs turning ‘running’ to ‘failed’ without any error report. Then I tried upload again, now it was stuck at “submitted” status again…

13

Hi @Kay1794, I fixed the issue with our evaluation worker and re-ran your submission. It is finished you should be able to see it now.

14

Got it. Thank you so much for your assistance @ram81!