[DBWorld] 2nd CfP - PERSPECTIVES 2021: Workshop on the Perspectives on the Evaluation of Recommender Systems

2nd Call for Papers and Abstracts
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

PERSPECTIVES 2021: Workshop on the Perspectives on the Evaluation of Recomm=
ender Systems =

Co-located with ACM RecSys 2021.
The workshop will take place online!

Workshop website: https://perspectives-ws.github.io/2021/
Paper submission: https://easychair.org/conferences/?conf=3Dperspectives202=
1 =


In this workshop, we aim to bring together researchers and practitioners fr=
om academia and industry to critically discuss the different, and maybe eve=
n diverging or contradictory perspectives on the evaluation of recommender =
systems.

Important Dates
	=E2=80=A2 Paper submission deadline: ***July 29th, 2021***
	=E2=80=A2 Author notification: August 21st, 2021
	=E2=80=A2 Camera-ready version deadline: September 4th, 2021
	=E2=80=A2 Workshop: Sat, 25 September 2021 (in scope of RecSys 2021)


Objectives
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

Evaluation is essential when conducting rigorous research in recommender sy=
stems (RS). It may span the evaluation of early ideas and approaches up to =
elaborate systems in operation; it may target a wide spectrum of different =
aspects being evaluated. Naturally, we do (and have to) take various perspe=
ctives on the evaluation of RS. Thereby, the term =E2=80=9Cperspective=E2=
=80=9D may, for instance, refer to various purposes of a RS, the various st=
akeholders affected by a RS, or the potential risks that ought to be minimi=
zed. Further, we have to consider that various methodological approaches an=
d experimental designs represent different perspectives on evaluation. The =
perspective on the evaluation of RS may also be substantially characterized=
 by the available resources. The access to resources will likely be differe=
nt for PhD students compared to established researchers in industry.

Acknowledging that there are various perspectives on the evaluation of RS, =
we want to put into discussion whether there is a ``golden standard'' for t=
he evaluation of RS, and---if so---if it indeed is ``golden'' in any sense.=
 We postulate that the various perspectives are valid and reasonable, and a=
im to reach out to the community to discuss and reason about them.

The goal of the workshop is to capture the current state of evaluation, and=
 gauge whether there is, or should be, a different target that RS evaluatio=
n should strive for. The workshop will address the question: where should w=
e go from here as a community? and aims at coming up with concrete steps fo=
r action.

We have a particularly strong commitment to invite and integrate researcher=
s at the beginning of their careers and want to equally integrate establish=
ed researchers and practitioners, from industry and academia alike. It is o=
ur particular concern to give a voice to the various perspectives involved.

Topics of Interest
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Topics of interest include, but are not limited to, the following:

	=E2=80=A2 Case studies of difficult, hard-to-evaluate scenarios
	=E2=80=A2 Evaluations with contradicting results
	=E2=80=A2 Showcasing (structural) problems in RS evaluation
	=E2=80=A2 Integration of offline and online experiments
	=E2=80=A2 Multi-Stakeholder evaluation
	=E2=80=A2 Divergence between evaluation goals and what is actually capture=
d by the evaluation
	=E2=80=A2 Nontrivial and unexpected experiences from practitioners

We deliberately solicit papers reporting problems and (negative) experience=
s regarding RS evaluation, as we consider the reflection on unsuccessful, i=
nadequate or insufficient evaluations as a fruitful source for yet another =
perspective on RS evaluation that can spark discussions at the workshop. Ac=
cordingly, submissions may also address the following
themes: =

(a) =E2=80=9Clessons learned=E2=80=9D from the successful application of RS=
 evaluation or from =E2=80=9Cpost mortem=E2=80=9D analyses describing speci=
fic evaluation strategies that failed to uncover decisive elements, =

(b) =E2=80=9Coverview papers=E2=80=9D analyzing patterns of challenges or o=
bstacles to evaluation, and =

(c) =E2=80=9Csolution papers=E2=80=9D presenting solutions for specific eva=
luation scenarios. =

Additionally, =

(d) =E2=80=9Cvisionary papers=E2=80=9D discussing novel and future evaluati=
on aspects will be considered as well.


Submissions
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
We solicit two forms of contributions: First, we solicit paper submissions =
that will undergo peer review. Accepted papers will be published and presen=
ted at the workshop. Second, we offer the opportunity to present ideas with=
out a paper submission. In this case, we call for the submission of abstrac=
ts that will be reviewed by the workshop organizers. Accepted abstracts wil=
l be presented at the workshop, but not published.

1) Paper Submissions

We solicit papers with 4 up to 10 pages. Along the lines of this year=E2=80=
=99s call for papers of the main conference, we do not distinguish between =
full and short (or position) papers. =


Submissions will undergo single-blind peer review by a minimum of three pro=
gram committee members and will be selected based on their quality, novelty=
, clarity, and relevance. Authors of accepted papers will be invited to pre=
sent their work during the workshop and will be published as open access wo=
rkshop proceedings via ceur-ws.org. At least one author of each accepted pa=
per must attend the workshop and present the work.


2) Abstract Submissions
We solicit abstracts with 200-350 words, to be submitted in EasyChair. The =
workshop organizers will select abstracts based on quality, clarity, releva=
nce, and their potential to spark interesting discussion during the worksho=
p. Authors of accepted abstracts will be invited to present their work duri=
ng the workshop. =



Workshop Organizers
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
	=E2=80=A2 Eva Zangerle (Universit=C3=A4t Innsbruck, Austria)
	=E2=80=A2 Christine Bauer (Utrecht University, The Netherlands)
	=E2=80=A2 Alan Said (University of Gothenburg, Sweden)



--  =

Dr. Christine Bauer | Assistant Professor | Department of Information and C=
omputing Sciences | Utrecht University | Princetonplein 5, 3584 CC Utrecht =
| Buys Ballotgebouw, room 4.13 | +31 30 253 4158 | [email protected] | www.uu.n=
l/staff/CBauer | https://christinebauer.eu | Present: Mon-Fri
_______________________________________________
Please do not post msgs that are not relevant to the database community at =
large.  Go to www.cs.wisc.edu/dbworld for guidelines and posting forms.
To unsubscribe, go to https://lists.cs.wisc.edu/mailman/listinfo/dbworld