ESWC 2014 Call for Challenge: Concept-Level Sentiment Analysis

--===============9141040609945783221==
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable


** apologies for cross-posting **

=3D=3D=3D=3D Call for Challenge: Concept-Level Sentiment Analysis =3D=3D=3D=
=3D

Challenge Website: http://challenges.2014.eswc-conferences.org/SemSA
Call Web page: http://2014.eswc-conferences.org/important-dates/call-SemS=
ATarg
				=09
MOTIVATION AND OBJECTIVES
Mining opinions and sentiments from natural language is an extremely diff=
icult task as it involves a deep understanding of most of the explicit an=
d implicit, regular and irregular, syntactical and semantic rules proper =
of a language. Existing approaches mainly rely on parts of text in which =
opinions and sentiments are explicitly expressed such as polarity terms, =
affect words and their co-occurrence frequencies. However, opinions and s=
entiments are often conveyed implicitly through latent semantics, which m=
ake purely syntactical approaches ineffective. To this end, concept-level=
 sentiment analysis aims to go beyond a mere word-level analysis of text =
and provide novel approaches to opinion mining and sentiment analysis tha=
t allow a more efficient passage from (unstructured) textual information =
to (structured) machine-processable data, in potentially any domain.

Concept-level sentiment analysis focuses on a semantic analysis of text t=
hrough the use of web ontologies or semantic networks, which allow the ag=
gregation of conceptual and affective information associated with natural=
 language opinions. By relying on large semantic knowledge bases, concept=
-level sentiment analysis steps away from blind use of keywords and word =
co-occurrence count, but rather relies on the implicit features associate=
d with natural language concepts.

This Challenge focuses on the introduction, presentation, and discussion =
of novel approaches to concept-level sentiment analysis. Participants wil=
l have to design a concept-level opinion-mining engine that exploits comm=
on-sense knowledge bases, e.g., SenticNet, and/or Linked Data and Semanti=
c Web ontologies, e.g., DBPedia, to perform multi-domain sentiment analys=
is. The main motivation for the Challenge, in particular, is to go beyond=
 a mere word-level analysis of natural language text and provide novel co=
ncept-level tools and techniques that allow a more efficient passage from=
 (unstructured) natural language to (structured) machine-processable data=
, in potentially any domain.

Systems must have a semantics flavor (e.g., by making use of Linked Data =
or known semantic networks within their core functionalities) and authors=
 need to show how the introduction of semantics can be used to obtain val=
uable information, functionality or performance. Existing natural languag=
e processing methods or statistical approaches can be used too as long as=
 the semantics plays a main role within the core approach (engines based =
merely on syntax/word-count will be excluded from the competition).


TARGET AUDIENCE
The Challenge is open to everyone from industry and academia.

TASKS
The Concept-Level Sentiment Analysis Challenge is defined in terms of dif=
ferent tasks. The first task is elementary whereas the others are more ad=
vanced. The input units of each task are sentences. Sentences are assumed=
 to be in grammatically correct American English and have to be processed=
 according to the input format specified at http://sentic.net/challenge/s=
entence.

Elementary Task: Polarity Detection
The main goal of the task is polarity detection. The proposed systems wil=
l be assessed according to precision, recall and F-measure of detected bi=
nary polarity values (1=3Dpositive; 0=3Dnegative) for each input sentence=
 of the evaluation dataset, following the same format as in http://sentic=
.net/challenge/task0. The problem of subjectivity detection is not addres=
sed within this Challenge, hence participants can assume that there will =
be no neutral sentences. Participants are encouraged to use the Sentic AP=
I or further develop and apply sentic computing tools.

Advanced Task #1: Aspect-Based Sentiment Analysis
The output of this task will be a set of aspects of the reviewed product =
and a binary polarity value associated to each of such aspects, in the fo=
rmat specified at http://sentic.net/challenge/task1. So, for example, whi=
le for the Elementary task an overall polarity (positive or negative) is =
expected for a review about a mobile phone, this task requires a set of a=
spects (such as =E2=80=98speaker', =E2=80=98touchscreen', =E2=80=98camera=
', etc.) and a polarity value (positive OR negative) associated with each=
 of such aspects. Systems will be assessed according to both aspect extra=
ction and aspect polarity detection.

Advanced Task #2: Semantic Parsing
As suggested by the title, the Challenge focuses on sentiment analysis at=
 concept-level. This means that the proposed systems are not supposed to =
work at word/syntax level but rather work with concepts/semantics. Hence,=
 this task will evaluate the capability of the proposed systems to decons=
truct natural language text into concepts, following the same format as i=
n http://sentic.net/challenge/task2. SenticNet will be taken as a referen=
ce to test the efficiency of the proposed parsers, but extracted concepts=
 won't necessary have to match SenticNet concepts. The proposed systems, =
for example, are supposed to be able to extract a multi-word expression l=
ike =E2=80=98buy christmas present' from sentences such as =E2=80=9CToday=
 I bought a lot of very nice Christmas presents'. The number of extracted=
 concepts per sentence will be assessed through precision, recall and F-m=
easure against the evaluation dataset.

Advanced Task #3: Topic Spotting
Input sentences will be about four different domains, namely: books, DVDs=
, electronics, and kitchen appliances. This task focuses on the automatic=
 classification of sentences into one of such domains, in the format spec=
ified at http://sentic.net/challenge/task3. All sentences are assumed to =
belong to only one of the above-mentioned domains. The proposed systems a=
re supposed to exploit the extracted concepts to infer which domain each =
sentence belongs to. Classification accuracy will be evaluated in terms o=
f precision, recall and F-measure against the evaluation dataset.

EVALUATION DATASET
Systems will be evaluated against a testing dataset which will be reveale=
d and released after the first-round of evaluation during the Conference.=
 The dataset will be made public on the challenge website. Participants a=
re suggested to train and/or test their own systems using the Blitzer Dat=
aset. The testing dataset will be constructed in the same way and from th=
e same sources as the Blitzer dataset.

EVALUATION
The evaluation will be performed by the members of the Program Committee.=
 For systems that can be tuned with different parameters, please indicate=
 a range of up to 4 sets of settings. Settings with the best F-measures w=
ill be considered for judgment. For each system, reviewers will give a nu=
merical score within the range [1-10] and details motivating their choice=
. The scores will be given to the following aspects:
1. Use of common-sense knowledge and semantics;
2. Precision, recall, and F-measure wrt the selected task;
3. Computational time;
4. Innovative nature of the approach.

JUDGING AND PRIZES
After a first round of review, the Program Committee and the chairs will =
select a number of submissions confirming to the challenge requirements t=
hat will be invited to present their work. Submissions accepted for prese=
ntation will be included in post-proceedings and will receive constructiv=
e reviews from the Program Committee. All accepted submissions will have =
a slot in a poster session dedicated to the challenge. In addition, the w=
inners will present their work in a special slot of the main program of E=
SWC and will be invited to submit a paper to a dedicated Semantic Web Jou=
rnal special issue.

For the Concept-Level Sentiment Analysis Challenge there will be two awar=
ds for each task:
* Quantitative: the system with the highest average score in items 1-3 ab=
ove;
* Innovative: the system with the highest score in item 4 above.
There will be a board of judges at the conference who will evaluate again=
 the systems in more detail. The judges will then meet in private to disc=
uss the entries and to determine the winners. It may happen that the same=
 system runs for both the awards.


HOW TO PARTICIPATE
The following information has to be provided:
* Abstract: no more than 200 words.
* Description: It should contain the details of the system, including why=
 the system is innovative, how it uses Semantic Web, which features or fu=
nctions the system provides, what design choices were made and what lesso=
ns were learned. The description should also summarize how participants h=
ave addressed the evaluation tasks. Papers must be submitted in PDF forma=
t, following the style of the Springer's Lecture Notes in Computer Scienc=
e (LNCS) series (http://www.springer.com/computer/lncs/lncs+authors), and=
 not exceeding 5 pages in length.
* Web Access: The application can either be accessible via the web or dow=
nloadable. If the application is not publicly accessible, password must b=
e provided. A short set of instructions on how to use the application sho=
uld be provided as well.

All submissions should be provided via EasyChair https://www.easychair.or=
g/conferences/?conf=3Deswc2014-challenges

Please share comments and questions with the challenge mailing list. The =
organizers will assist you for any potential issues that could be raised.


MAILING LIST
We invite the potential participants to subscribe to our mailing list in =
order to be kept up to date with the latest news related to the challenge=
.=20

https://lists.sti2.org/mailman/listinfo/eswc2014-semsa-challenge


IMPORTANT DATES
* March 7, 2014, 23:59 (Hawaii time): Abstract Submission=20
* March 14, 2014, 23:59 (Hawaii time): Submission=20
* April 9, 2014, 23:59 (Hawaii time): Notification of acceptance
* May 27-29, 2014: Challenge days

CHALLENGE CHAIRS
* Erik Cambria (National University of Singapore, SG)
* Diego Reforgiato Recupero (CNR STLAB Laboratory, IT)

PROGRAM COMMITTEE
* Newton Howard (MIT Media Laboratory, US)
* ChengXiang Zhai (University of Illinois at Urbana-Champaign, US)
* Rada Mihalcea (University of North Texas, US)
* Ping Chen (University of Houston-Downtown, US)
* Yongzheng Zhang (LinkedIn Inc., US)
* Giuseppe Di Fabbrizio (Amazon Inc., US)
* Rui Xia (Nanjing University of Science and Technology, CN)
* Rafal Rzepka (Hokkaido University, JP)
* Amir Hussain (University of Stirling, UK)
* Alexander Gelbukh (National Polytechnic Institute, MX)
* Bjoern Schuller, (Technical University of Munich, DE)
* Amitava Das (Samsung Research India, IN)
* Dipankar Das (National Institute of Technology, IN)
* Carlo Strapparava (Fondazione Bruno Kessler, IT)
* Stefano Squartini (Marche Polytechnic University, IT)
* Cristina Bosco (University of Torino, IT)
* Paolo Rosso (Technical University of Valencia, ES)

ESWC CHALLENGE COORDINATOR
* Milan Stankovic (Sepage & Universite Paris-Sorbonne, FR)


--===============9141040609945783221==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
protege-discussion mailing list
[email protected]
https://mailman.stanford.edu/mailman/listinfo/protege-discussion

Instructions for unsubscribing: http://protege.stanford.edu/doc/faq.html#01a.03

--===============9141040609945783221==--