Submission Returns "Killed" Message

Good morning,

I have had my last few submission attempts return a “Killed” message when failing. I had not had this issue before, it is only a recent problem I have encountered. Is there anything I am doing incorrectly?

Hi AXOLTL_LLC,

One of the test data files has been removed, causing the file names in the test data directory to no longer be consecutive. This change might be the reason for your submission attempts failing.

Are you retrieving the file list from the directory before loading the test data?

Best,
Peng

I think it is a memory issue, I added print statements around the data loading and see that it effectively loads the data. Each one of my models is about 4.5 GB and it fails when loading them into the pipeline.

Hi AXOLTL_LLC,

After reviewing your error output file and noting that your largest individual model file size was 14.5GB, we concur with your hypothesis that the issue is likely due to insufficient RAM. Please note the following computational restrictions for your future submissions:

  • CPU : 4 cpu cores will be available to you.
  • RAM : Approx. 16Gb of memory will be available.
  • GPU : 1 GPU, with 16Gb of memory will be available.
  • Run Time : Your submission should run within approx 10 minutes or less. If submissions take too long to run, they will fail.