Regarding the evaluation metrics, please find our comments below:
Our interest in designing the measure is to meet some practical considerations, ie. any model less than or equal to 1MB in model size, is good enough for our purposes.
Amongst all the models that satisfy 1MB or less model-size constraint, mIOU becomes the key deciding factor.
======== Notification of Paper Submission ========
This is a notification that the deadline of paper submission to ICCV workshop regarding OpenEDS challenges has changed.
Please note that there is a two tier submission, and the major difference with early deadline is that the accepted paper will appear in ICCV workshop proceedings. See table below for the two tiers.
|Tier 1||Tier 2|
|Paper submission deadline||Aug 19th||Aug 31st|
|Paper acceptance notification deadline||Aug 25th||Sept 15th|
|Camera Ready Deadline||Aug 30th||Sept 27th|
|Publication||* ICCV workshop proceedings||* Archived on IEEE Xplore|
|* Archived on IEEE Xplore||* Archived on CVF open access|
|* Archived on CVF open access|
PST time zone for all
OpenEDS Challenge Team
======== Common questions regarding paper submission ========
The challenge deadline remains the same, , which will close on Sept 15th.
Deadline to submit paper is stated in above table.
Specifically, (1) the first deadline is for folks who want to submit manuscript and are interested in seeing their paper (if accepted) appear in ICCV workshop proceedings; (2) the second deadline is for folks who are not ready to submit paper or do not necessarily care about the paper appearing in ICCV workshop proceedings.
For challenge participants they are welcome to submit papers to meet either of the two deadlines.
For winners, we expect the authors to submit paper, which are considered accepted, i.e the challenge winning papers will not undergo formal review if submitted after the challenge ends.
======== How to submit paper ========
- Please submit paper at: OpenEDS2019. This link is also provided on the official Workshop page under Submissions, https://research.fb.com/programs/the-2019-openeds-workshop-eye-tracking-for-vr-and-ar/
OpenEDS Challenge Team
- How many top positions will get the chance to present the paper.
- If we will submit the paper after the end of challenge, will it be considered as ‘tier1’ paper or ‘tier 2’ paper ?
- What would be the deadline for submitting the paper after the end of challenge.
Please see response below to your questions:
All submitted papers will be considered for publication. Accepted papers will also have an opportunity to present.
With regards to challenge specific submissions, only the top spot winner will be asked to present their work.
Papers submitted after tier 1 deadline, which is Aug 19th will all be considered for tier 2 submissions as long as they meet the Aug 31st submission deadline.
Only the top place winner will have the opportunity to submit their camera ready version of paper after the challenge deadline.
Might there be any opportunity to extend the Aug 19th deadline by 1 day?
We have been having trouble submitting our latest model over the last few days…
I’m currently in the first place and will hopefully keep it till the end, so I wonder about this point: I can’t submit a paper via link provided, since it won’t allow to submit past paper registration deadline, which was March 15, 2019.
Anything I can/should do?
Thanks for reaching out to us. If you are indeed the winner, we will officially announce the winners and reach out to them individually requesting them to submit their camera ready version of the manuscript.
Also winners are expected to submit their code base for evaluation.
Could you tell me extract time zone for tier 2 submission deadline. I almost completed the paper. Still few things need to be completed.
It’s PST time zone.
I am trying to upload the paper in the CMT link shared the page.
Yesterday i saw the paper submission domain names but today there is no button or domain for uploading the paper. What should i do ? Please help me
Just to confirm since it doesn’t say so explicitly anywhere in the rules or discussions, but common tactics for prediction improvement like ensembling and test-time augmentation aren’t allowed since they defeat the purpose of producing faster predictions, right?
Yes, correct. Since the purpose is both good accuracy and low compute cost.
Does this mean no pre-processing like normalization, cropping, filtering and other similar techniques can be applied on test images?
You could do that. But keep in mind that we also consider compute score as one of the metric
How would that compute score be calculated?
It’s stated in EvalAI semantic segmentation challenge, evaluation part. Also, available in https://research.fb.com/programs/openeds-challenge/
How many top positions will get the chance to present the paper?
openeds ans: Only the top place winner will have the opportunity to submit their camera-ready version of paper after the challenge deadline.
It means the top three or top one will be invited for presentation.