Covid-19 MLIA @ Eval - Call for Participation in Round 2

Covid-19 MLIA @ Eval
Community Evaluation Effort for MultiLingual Information Access (MLIA) on Covid-19
******* ROUND II ******* 

Call for Participation in the Second Evaluation Round 
Runs due by 15 September-15 October 2021

Website: http://eval.covid19-mlia.eu/ <http://eval.covid19-mlia.eu/>
Registration: http://eval.covid19-mlia.eu/registration/ <http://eval.covid19-mlia.eu/registration/>
Twitter: @covid19mlia
Email: [email protected]

------------------------------------
Overall Description
Covid-19 MLIA @ Eval organizes a community evaluation effort aimed at accelerating the creation of resources and tools for improved MultiLingual Information Access (MLIA) in the current emergency situation with a reference to a general public use case:

“Sofia has heard that a drug has been experimented in different countries and she would like to have a consolidated and trustworthy view of the main findings, whether the drug is effective or not, and whether there are any adverse effects”.

Distillation for the general public also implies a level of specialist-non-specialist communication, when the aggregated sources contain both disseminative and specialised sources. Therefore, the general public would need to understand medical expertise by using their correspondent in the "everyday " language or by using an appropriately calibrated language for the communication to be effective.

Covid-19 MLIA @ Eval adopts an incremental and iterative evaluation methodology to enable the release of intermediate (but functional) resources and to progressively (iteration-after-iteration) move towards finally consolidated tools and resources. We plan for three rounds, tentatively one and half-month long each.

Each round consists of the following phases:
data release
submission of participants’ runs
ground-truth creation (when needed)
scoring of participants’ runs
writing of the rolling technical report
round wrap virtual meeting

An integral part of the Covid-19 MLIA @ Eval approach is the sharing of information and ideas among the participants. This happens via the rolling technical report where participants describe their solutions as the evaluation rounds progress and the virtual meetings at the end of each round where participants briefly present the main highlights of what worked and what did not work and interactively discuss together in order to share ideas and improve for the next round.

Moreover, to facilitate the exchange of resources and components and to jointly work together for improving MLIA technologies for Covid-19, participants are provided with a dedicated git repository where to push and share the outcomes of your participation in the different rounds, i.e. runs, code, (language) resources, and a technical report. To facilitate sharing and re-use, all the contents of the repositories are released under the Creative Commons Attribution-ShareAlike 4.0 International License.
Tasks
Covid-19 MLIA @ Eval offered the following tasks for its first round and will continue for this second round
Information Extraction
More information at: http://eval.covid19-mlia.eu/task1/ <http://eval.covid19-mlia.eu/task1/>
Multilingual Semantic Search
More information at: http://eval.covid19-mlia.eu/task2/ <http://eval.covid19-mlia.eu/task2/>
Machine Translation
More information at: http://eval.covid19-mlia.eu/task3/ <http://eval.covid19-mlia.eu/task3/>
We offer tasks in the following languages: Arabic, English, French, German, Greek, Italian, Spanish, Swedish.

Registration for the tasks is open at:

http://eval.covid19-mlia.eu/registration/ <http://eval.covid19-mlia.eu/registration/>
Important Dates -  Second Round
Registration Opens: 14 June 2021
Second Round Starts: 21 June 2021 (data release)
Submission of Runs: 15 September - 15 October 2021 (depending on task)
Submission of Technical Report: 19 November 2021
Round II Wrap-up Virtual Meeting: 30 November-2 December 2021 (exact date TBD)
Second Round Ends: 2 December 2021

More detailed dates are provided on the Web pages dedicated to each specific task, since the internal scheduling of each task may differ slightly to better match the task design.
Organizers
Overall
Khalid Choukri, ELDA, France
Nicola Ferro, University of Padua, Italy

Data Acquisition and Engineering
Miltos Deligiannis, ILSP/Athena RC, Greece
Marwa Hadj Salah, ELRA/ELDA, France
Guillaume Jacquet, JRC, Italy
Vassilis Papavassiliou, ILSP/Athena RC, Greece
Stelios Piperidis, ILSP/Athena RC, Greece
Prokopis Prokopidis, ILSP/Athena RC, Greece


Information Extraction
Cyril Grouin, LISN, France
Thierry Declerck, DFKI, Germany
Pierre Zweigenbaum, LISN, France

Multilingual Semantic Search
Giorgio Maria Di Nunzio, University of Padua, Italy
Maria Eskevich, CLARIN ERIC

Machine Translation
Francisco Casacuberta, Universitat Politècnica de València, Spain
Miguel Domingo, Universitat Politècnica de València, Spain
Mercedes García-Martínez, Pangeanic, Spain
Manuel Herranz, Pangeanic, Spain
Supporters 
Covid-19 MLIA @ Eval is an evaluation effort promoted by several communities which are closely working together.

European Commission
https://ec.europa.eu/ <https://ec.europa.eu/>
European Language Resources Coordination (ELRC), 
http://www.lr-coordination.eu/ <http://www.lr-coordination.eu/>
European Language Resources Association (ELRA)
http://www.elra.info/en/ <http://www.elra.info/en/>
European Research Infrastructure for Language Resources and Technology (CLARIN)
https://www.clarin.eu/ <https://www.clarin.eu/>
CLEF Initiative
http://www.clef-initiative.eu/ <http://www.clef-initiative.eu/>
We hope that other communities will join us in this shared effort. You are very welcome!
Additional Information
You can find additional information on our Website:

http://eval.covid19-mlia.eu/ <http://eval.covid19-mlia.eu/>
You can follow updates through our Twitter account

@covid19mlia

Any requests or comments? Drop us an email at:
[email protected]


########################################################################

To unsubscribe from the IR list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=IR&A=1

This message was issued to members of www.jiscmail.ac.uk/IR, a mailing list hosted by www.jiscmail.ac.uk, terms & conditions are available at https://www.jiscmail.ac.uk/policyandsecurity/