I want to make MSCOCO challenge in EvalAI.
I put “captions_val2014.json” data in “annotations” folder.
“caption_val2014.json” is data for evaluation.
And installed “pycocotools” & “pycocoevalcap” module.
The code below is “main.py” that is in “evaluation_script” folder.
from pycocotools.coco import COCO
from pycocoevalcap.eval import COCOEvalCap
import json
def evaluate(test_annotation_file, user_submission_file, phase_codename):
output = {}
with open(user_submission_file, "r") as user_js:
preds = json.load(user_js)
coco = COCO(test_annotation_file)
valids = coco.getImgIds()
cocoRes = coco.loadRes(user_submission_file)
cocoEval = COCOEvalCap(coco, cocoRes)
cocoEval.params['image_id'] = cocoRes.getImgIds()
cocoEval.evaluate()
output['result'] = [
{
'test_split': {
}
}
]
print("Evaluating for Test Phase")
for metric, score in cocoEval.eval.items():
output['result'][0]['test_split'][metric] = float(score)
output["submission_result"] = output["result"][0]["test_split"]
print("Completed evaluation for Test Phase")
return output
This is my submission page
and I got an Error in “Stderr file” category…
Traceback (most recent call last):
File "/code/scripts/workers/submission_worker.py", line 480, in run_submission
submission_metadata=submission_serializer.data,
TypeError: evaluate() got an unexpected keyword argument 'submission_metadata'
I don’t know how to solve this error…