Call for Track proposals - Forum for Information Retrieval Evaluation 2026 - Deadline extended - fire.irsi.org.in

Dear Colleagues,

Call for Evaluation and Benchmarking Track 2026 is extended until April 15th

https://easychair.org/conferences/?conf=fire2026

----------------------------------------------------------------------------------------------------

We invite proposals for the Evaluation and Benchmarking Track at FIRE 2026.

FIRE 2026 is the 18th edition of the annual meeting of Forum for 
Information Retrieval Evaluation (fire.irsi.org.in 
<https://fire.irsi.org.in>). Since its inception in 2008, FIRE had a 
strong focus on shared tasks similar to those offered at Evaluation 
forums like TREC, CLEF, and NTCIR. The shared tasks should focus on 
solving specific problems in the area of information access and, more 
importantly, on generating high quality evaluation datasets for the 
research community. In line with this objective, this year the FIRE 
tracks emphasize selecting tasks that either introduce a new paradigm in 
the field or generate a substantial amount of valuable benchmark data 
that can support future research and experimentation.

It is not required for the tasks to focus on a specific language, and 
they can broadly cover any problem in the fields related (but not 
limited) to IR, NLP, multi-modal information access, and ML. However, 
the organizers especially encourage proposals for tracks related to 
South Asian, African, and Middle Eastern languages. In the past, FIRE 
has hosted tracks from Arabic, Persian, German, Russian, and Urdu 
languages besides several Indian languages. We aim to continue these 
efforts and include more language groups from these regions. For knowing 
more about tracks in past FIRE meetings, you can visit fire.irsi.org.in 
<https://fire.irsi.org.in>

Informal inquiries can also be sent to the track chairs.

Please include the following details in your proposal:

1. Track name
2. Track description
3. Use case/s
4. Target Audience and number of expected submissions
5. Data(*) (Fair Details)
6. Evaluation plan
7. Timeline: Please try to align with the FIRE conference dates as given 
below
8. Organizer/s Details
9. Prior experience in organizing shared task/workshop at relevant venues


*Tentative Timeline*
*15th April, 2026* 	Track proposals due
*24th April, 2026* 	Track acceptance notification
*15th May, 2026* 	Open track websites and release of training data
*15th June, 2026* 	Test data release
*30th June, 2026* 	Run submission deadline
*15th July, 2026* 	Track results declaration
*30th August, 2026* 	Working notes due
*30th September, 2026* 	Camera-ready copies of working notes and 
overview paper due
*December, 2026 - Dates TBD* 	FIRE 2026 Conference


Please submit your proposals via Easychair: 
https://easychair.org/conferences/?conf=fire2026

(*) We require that after FIRE, the data should be made publicly 
available through Information Retrieval Society of India. In case, data 
can not be distributed publicly (e.g., Twitter data), a unique 
identifier that can be used to recreate the original corpus can be 
provided (e.g., tweet ids in case of Twitter data). This disbursal will 
be governed by a copyright form, which the users have to sign before 
getting the data. A sample form is available at ( 
fire.irsi.org.in/fire/static/data 
<https://fire.irsi.org.in/fire/static/data> ).

In case, it is not preferable/possible for the track organizers to share 
the data, please mention this in the proposal with specific concerns. 
Exceptions can be made for tracks where data from industry is used or in 
case of other serious legal or ethical concerns.

The aim of organizing these tracks at FIRE is to have debates and 
discussions on focused topics and give feedback to participants. As a 
result, at least one of the track organizers from each track is expected 
to attend FIRE and present the overview of track in person. In case of 
non-attendance of any of the organizers, the team will not be allowed to 
offer a track next year.

We will try to provide student volunteers for support in the proposed 
tracks. They will basically be undergraduate students interested in IR 
and related fields and can help with corpus creation, evaluation, 
correspondence with participants, etc. If you require any such support, 
kindly mention that in the track proposals along with the number of 
students required.

Hoping to have an enthusiastic response from your end.

*Overall Track Coordinators, FIRE 2026*

Thomas Mandl (Universitat Hildesheim, Germany)
Kripabandhu Ghosh (IISER Kolkata, India)
Srijoni Majumdar (University of Leeds, UK)

_______________________________________________
Corpora mailing list -- [email protected]
https://list.elra.info/mailman3/postorius/lists/corpora.list.elra.info/
To unsubscribe send an email to [email protected]