Call for Participation: TREC 2022 Fair Ranking Track

---------------------------------------------------------------------

Call for Participation:  TREC 2022 Fair Ranking Track

---------------------------------------------------------------------



Training queries, corpus, metadata and participant instructions are now available at the track webpage: https://fair-trec.github.io



The TREC Fair Ranking Track aims to provide a platform for participants to develop and evaluate novel retrieval algorithms that can provide a fair exposure to a mixture of demographics or attributes, such as gender, that are represented by relevant documents in response to a search query. For example, particular demographics or attributes can be represented by the documents’ topical content or authors.



The 2022 Fair Ranking Track is focused on supporting Wikipedia editors who are looking to improve the Wikipedia’s coverage of WikiProject topics. WikiProject coordinators/editors search for Wikipedia documents that are relevant to a WikiProject and that are in need of editing to improve the quality of the articles. The objective of the track is to ensure that documents that are about, or somehow represent, certain protected characteristics receive a fair exposure to the Wikipedia coordinators/editors, so that the characteristics have an equal opportunity of being well represented in Wikipedia.



This year, the Fair Ranking Track focuses on intersectionality fairness. There are many different sets, or groups, of protected characteristics that can hold a distinct point of view as to whether a ranking is fair. Moreover, a document could possibly represent many different protected characteristics. This leads to many interesting fairness related questions, such as “Does optimising a ranking to be fair to many protected characteristics result in a system that is unfair to particular protected characteristics?” and “Are similar protected characteristics treated similarly by particular fairness strategies?”. The TREC 2022 Fair Ranking Track evaluation will focus on how fairly systems treat many different protected characteristics and the impacts for particular subsets of those characteristics.



Timeline:

  *   May 2022: guidelines released.
  *   June 2022: training queries and corpus released
  *   July 2022: evaluation queries released
  *   August 2022: submissions due
  *   September 2022: evaluated submissions returned



For more information or to ask questions, join the track’s google group: https://groups.google.com/d/forum/fair-trec


Organisers:

  *   Michael Ekstrand: Boise State University
  *   Graham McDonald: University of Glasgow
  *   Amifa Raj: Boise State University
  *   Isaac Johnson: Wikimedia Foundation


########################################################################

To unsubscribe from the IR list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=IR&A=1

This message was issued to members of www.jiscmail.ac.uk/IR, a mailing list hosted by www.jiscmail.ac.uk, terms & conditions are available at https://www.jiscmail.ac.uk/policyandsecurity/