Minival phase submission failed

Hi, I submitted a docker (the agent is not the baseline provided). It was submitted successfully, but the running result is failed. The stderr doesn’t show any failure related message. the stdout shows that the eval is finished and succ is 0 and spl is 0. May I ask why the submission is failed?

Hi Shupeng,

Thanks for submitting!! I checked your submission file and /opt/GibsonEnvV2 is not up to date, you are on commit 37c1fbeadc6e03634f15c3a2598ebb6df653686d for March 4 and we would need 3453531543e7afa9e4e375dcb16e3e151c72d09d on March 7. If you update that, the minival submission should pass. (the numbers doesn’t matter too much, so don’t worry) Our gibsonchallenge/gibsonv2:latest after March 7 contains the latest version, and looks like you are using an old one before March 4.

Best,
Fei

You stdout and stderr file looks fine. So you don’t need to retrain the policy, just update /opt/GibsonEnvV2 with git pull origin gibson_sim2real would be enough.

I updated the gibsonEnvV2, and the submission is good now.
Thank you.
Shupeng

Great, don’t worry too much about the performance numbers, as minival only evaluate 10 times and results might not be statistically meaningful. This is mainly for sanity checking submissions actually go through. We will be opening the simulation evaluation soon.

OK, good.
Just come up with another unrelated question: I was using the “done” information from the environment as observation to the agent. It means I did minor modification to the challenge.py file :
action = gent.act(state, done) #[it was action = gent.act(state)]
Can I pass the binary variable “done” to the agent? Or the agent should be aware of when it reach the target?
Thank you,
Shupeng

Hello team,

I tried submitting to Dev phase today but got the error - Error: Sorry, cannot accept submissions since challenge phase is not active. I had made some changes to my policy to speed it up. I wanted to submit so I can check the execution time of my policy in your hardware. I feel that my hardware is taking more time. Is there an easier way to do it instead of clogging the other phases?

Hi @Ruddra_dev,

I have extended the Dev phase deadline so you should be able to submit now. Thanks!

Thank you @chengshu, I was able to make a few submissions. Recently I got an error : no space left on device. Is this something you can help with? Or guide me in bypassing this error in future?

This is caused by the evaluation machine running out of space, I can manually fix it.

@Ruddra_dev fixed, you can try again.