Investigating and Mitigating Biases in Crowdsourced Data - CSCW 21 Workshop - Call for Papers

*CSCW 21 Workshop - Investigating and Mitigating Biases in Crowdsourced
Data
<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsites.google.com%2Fview%2Fbiases-in-crowdsourced-data&data=04%7C01%7Cgkazai%40microsoft.com%7C9d3bcab5c6d34dc5e0dd08d9637022be%7C72f988bf86f141af91ab2d7cd011db47%7C0%7C0%7C637650155585291342%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=FpAhBZIJ40mrxP2It73HluPHcEih%2BkUQNf6znugiEyQ%3D&reserved=0>*



The workshop will be held at ACM CSCW 2021, virtually on the *23rd of
October 2021* from 3 PM to 8 PM EDT.



The workshop will explore how specific crowdsourcing workflows, worker
attributes, and work practices contribute to biases in data. We also plan
to discuss research directions to mitigate labelling biases, particularly
in a crowdsourced context, and the implications of such methods for the
workers.



We invite participants to take part in the workshop challenge and/or submit
a position paper.

   - Submit a short position paper by *10 September 2021*
   - Register for the Crowd Bias Challenge by 23 September 2021


*Workshop Themes*

   - Understanding how annotator attributes contribute to biases

Research on crowd work has often focused on task accuracy whereas other
factors such as biases in data have received limited attention. We are
interested in reviewing existing approaches and discussing ongoing work
that helps us better understand annotation attributes contributing to
biases.

   - Quantifying bias in annotated data

An important step towards bias mitigation is detecting such biases and
measuring the extent of biases in data. We seek to discuss different
methods, metrics and challenges in quantifying biases, particularly in
crowdsourced data. Further, we are interested in ways of comparing biases
across different samples and investigating if specific biases are
task-specific or task-independent.

   - Novel approaches to mitigate crowd bias

We plan to explore novel methods that aim to reduce biases in crowd
annotation in particular. Current approaches range from worker
pre-selection, improving task presentation and dynamic task assignment. We
seek to discuss shortcomings and limitations of existing and ongoing
approaches and ideate future directions.

   - Impact on crowd workers

We want to explore how bias identification and mitigation strategies can
impact the actual workers, positively or negatively. For example, workers
in certain groups may face increased competition and lack of task
availability. Collecting worker attributes and profiling could raise
ethical concerns.





More details at https://sites.google.com/view/biases-in-crowdsourced-data
<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsites.google.com%2Fview%2Fbiases-in-crowdsourced-data&data=04%7C01%7Cgkazai%40microsoft.com%7C9d3bcab5c6d34dc5e0dd08d9637022be%7C72f988bf86f141af91ab2d7cd011db47%7C0%7C0%7C637650155585301295%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=LYKeZeYweYAItJdQi7IvArAvNq3fi1ibKqrqVR4i8kc%3D&reserved=0>

########################################################################

To unsubscribe from the IR list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=IR&A=1

This message was issued to members of www.jiscmail.ac.uk/IR, a mailing list hosted by www.jiscmail.ac.uk, terms & conditions are available at https://www.jiscmail.ac.uk/policyandsecurity/