[CALL FOR PARTICIPATION] Long-Form Analogy Evaluation Challenge @ INLG 2026
Bhavya - via Corpora <[email protected]> Mon, 6 Jul 2026 01:51:17 +0000
| Newsgroups | gmane.science.linguistics.corpora |
|---|---|
| Message-ID | <SA1PR15MB496778BC0DD7CD8D2917F1E5D6F12@SA1PR15MB4967.namprd15.prod.outlook.com> |
--===============5557789845951762466== Content-Language: en-US Content-Type: multipart/alternative; boundary="_000_SA1PR15MB496778BC0DD7CD8D2917F1E5D6F12SA1PR15MB4967namp_" --_000_SA1PR15MB496778BC0DD7CD8D2917F1E5D6F12SA1PR15MB4967namp_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Dear colleagues, We are pleased to announce the Long-Form Analogy Evaluation Challenge, a sh= ared task focused on automatically evaluating generated analogies for expla= ining Computer Science concepts. The shared task will be co-located with IN= LG 2026 (https://2026.inlgmeeting.org/), and participants will have the opp= ortunity to write a paper describing their system and approach. Submission Deadline: August 1, 2026 Prizes: The top 3 participants will receive $100, $50, and $25, respectivel= y. About the Challenge Participants are asked to develop a system that predicts human quality rati= ngs for long-form analogies. Given a concept, its description, a textual an= alogy, and an accompanying animation video, systems must predict scores acr= oss textual and video-based criteria. Submissions are evaluated using rank = correlation with human judgments. Links * Competition: https://huggingface.co/spaces/analogy-evaluation/Analogy= -Evaluation-Challenge * Dataset: https://huggingface.co/datasets/analogy-evaluation/challenge= -dataset * Baseline code: https://github.com/yurinoviello/analogy-eval-baseline-= INLG We warmly encourage members of this community to participate and share this= announcement with colleagues and students who may be interested. Best regards, Bhavya Research Scientist, IBM Yuri Noviello PhD Student, Delft University of Technology --_000_SA1PR15MB496778BC0DD7CD8D2917F1E5D6F12SA1PR15MB4967namp_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable <html> <head> <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"= > </head> <body> <p style=3D"text-align: left; text-indent: 0px; text-transform: none; backg= round-color: rgb(255, 255, 255);"> <span style=3D"font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt= ; color: rgb(34, 34, 34);">Dear colleagues,</span></p> <p style=3D"text-align: left; text-indent: 0px; text-transform: none; backg= round-color: rgb(255, 255, 255);"> <span style=3D"font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt= ; color: rgb(34, 34, 34);">We are pleased to announce the Long-Form Analogy= Evaluation Challenge, a shared task focused on automatically evaluating ge= nerated analogies for explaining Computer Science concepts. The shared task will be co-located with INLG 2026 (</spa= n><span style=3D"font-family: Arial, Helvetica, sans-serif; font-size: 9.75= pt; color: rgb(17, 85, 204);"><a href=3D"https://2026.inlgmeeting.org/" tar= get=3D"_blank" title=3D"https://2026.inlgmeeting.org/" data-saferedirecturl= =3D"https://www.google.com/url?q=3Dhttps://2026.inlgmeeting.org/&source= =3Dgmail&ust=3D1783388809249000&usg=3DAOvVaw2SQX1ATzGH9B4VGl8TQ-Ee"= data-outlook-id=3D"99676801-67b8-425b-ae97-0b7ea180d051" data-ogsc=3D"rgb(= 17, 85, 204)" style=3D"color: rgb(17, 85, 204); margin-top: 0px; margin-bot= tom: 0px;">https://2026.inlgmeeting.org/</a></span><span style=3D"font-fami= ly: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34)= ;"><wbr data-ogsc=3D"rgb(34, 34, 34)" style=3D"color: rgb(34, 34, 34); back= ground-color: inherit;">), and participants will have the opportunity to write a paper describing the= ir system and approach.</span></p> <p style=3D"text-align: left; text-indent: 0px; text-transform: none; backg= round-color: rgb(255, 255, 255);"> <span style=3D"font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt= ; color: rgb(34, 34, 34);"><b>Submission Deadline:</b> August 1, 2026<= /span></p> <p style=3D"text-align: left; text-indent: 0px; text-transform: none; backg= round-color: rgb(255, 255, 255);"> <span style=3D"font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt= ; color: rgb(34, 34, 34);"><b>Prizes:</b> The top 3 participants will = receive $100, $50, and $25, respectively.</span></p> <p style=3D"text-align: left; text-indent: 0px; text-transform: none; backg= round-color: rgb(255, 255, 255);"> <span style=3D"font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt= ; color: rgb(34, 34, 34);"><b>About the Challenge</b></span></p> <p style=3D"text-align: left; text-indent: 0px; text-transform: none; backg= round-color: rgb(255, 255, 255);"> <span style=3D"font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt= ; color: rgb(34, 34, 34);">Participants are asked to develop a system that = predicts human quality ratings for long-form analogies. Given a concept, it= s description, a textual analogy, and an accompanying animation video, systems must predict scores across te= xtual and video-based criteria. Submissions are evaluated using rank correl= ation with human judgments.</span></p> <p style=3D"text-align: left; text-indent: 0px; text-transform: none; backg= round-color: rgb(255, 255, 255);"> <span style=3D"font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt= ; color: rgb(34, 34, 34);"><b>Links</b></span></p> <ul data-ogsb=3D"rgb(255, 255, 255)" style=3D"text-align: left; background-= color: rgb(255, 255, 255);"> <li style=3D"font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt; = color: rgb(34, 34, 34); margin-left: 15px;"> <span style=3D"text-transform: none;" role=3D"presentation">Competition: </= span><span style=3D"color: rgb(17, 85, 204); text-transform: none;" role=3D= "presentation"><a href=3D"https://huggingface.co/spaces/analogy-evaluation/= Analogy-Evaluation-Challenge" target=3D"_blank" title=3D"https://huggingfac= e.co/spaces/analogy-evaluation/Analogy-Evaluation-Challenge" data-saferedir= ecturl=3D"https://www.google.com/url?q=3Dhttps://huggingface.co/spaces/anal= ogy-evaluation/Analogy-Evaluation-Challenge&source=3Dgmail&ust=3D17= 83388809249000&usg=3DAOvVaw2Uqk0HDTgbuqQl4bZJJwUI" data-outlook-id=3D"3= e96e815-f85b-4250-b5f2-2ac009a0581a" data-ogsc=3D"rgb(17, 85, 204)" style= =3D"color: rgb(17, 85, 204);">https://huggingface.co/spaces/<wbr data-ogsc= =3D"rgb(17, 85, 204)" style=3D"color: rgb(17, 85, 204); background-color: i= nherit;">analogy-evaluation/Analogy-<wbr data-ogsc=3D"rgb(17, 85, 204)" sty= le=3D"color: rgb(17, 85, 204); background-color: inherit;">Evaluation-Chall= enge</a></span></li><li style=3D"font-family: Arial, Helvetica, sans-serif;= font-size: 9.75pt; color: rgb(34, 34, 34); margin-left: 15px;"> <span style=3D"text-transform: none;" role=3D"presentation">Dataset: </span= ><span style=3D"color: rgb(17, 85, 204); text-transform: none;" role=3D"pre= sentation"><a href=3D"https://huggingface.co/datasets/analogy-evaluation/ch= allenge-dataset" target=3D"_blank" title=3D"https://huggingface.co/datasets= /analogy-evaluation/challenge-dataset" data-saferedirecturl=3D"https://www.= google.com/url?q=3Dhttps://huggingface.co/datasets/analogy-evaluation/chall= enge-dataset&source=3Dgmail&ust=3D1783388809249000&usg=3DAOvVaw= 1MMyfkuLtv0KU-KEPbvowN" data-outlook-id=3D"88d1a215-6975-4da6-a694-12ac9ede= 42f8" data-ogsc=3D"rgb(17, 85, 204)" style=3D"color: rgb(17, 85, 204);">htt= ps://huggingface.co/<wbr data-ogsc=3D"rgb(17, 85, 204)" style=3D"color: rgb= (17, 85, 204); background-color: inherit;">datasets/analogy-evaluation/<wbr= data-ogsc=3D"rgb(17, 85, 204)" style=3D"color: rgb(17, 85, 204); backgroun= d-color: inherit;">challenge-dataset</a></span></li><li style=3D"font-famil= y: Arial, Helvetica, sans-serif; font-size: 9.75pt; color: rgb(34, 34, 34);= margin-left: 15px;"> <span style=3D"text-transform: none;" role=3D"presentation">Baseline code: = </span><span style=3D"color: rgb(17, 85, 204); text-transform: none;" role= =3D"presentation"><a href=3D"https://github.com/yurinoviello/analogy-eval-b= aseline-INLG" target=3D"_blank" title=3D"https://github.com/yurinoviello/an= alogy-eval-baseline-INLG" data-saferedirecturl=3D"https://www.google.com/ur= l?q=3Dhttps://github.com/yurinoviello/analogy-eval-baseline-INLG&source= =3Dgmail&ust=3D1783388809249000&usg=3DAOvVaw0VvcdfyV0-uDViggw2gIAH"= data-outlook-id=3D"11e5ef03-652a-4a5e-9830-0a596de7c4ce" data-ogsc=3D"rgb(= 17, 85, 204)" style=3D"color: rgb(17, 85, 204);">https://github.com/<wbr da= ta-ogsc=3D"rgb(17, 85, 204)" style=3D"color: rgb(17, 85, 204); background-c= olor: inherit;">yurinoviello/analogy-eval-<wbr data-ogsc=3D"rgb(17, 85, 204= )" style=3D"color: rgb(17, 85, 204); background-color: inherit;">baseline-I= NLG</a></span></li></ul> <p style=3D"text-align: left; text-indent: 0px; text-transform: none; backg= round-color: rgb(255, 255, 255);"> <span style=3D"font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt= ; color: rgb(34, 34, 34);">We warmly encourage members of this community to= participate and share this announcement with colleagues and students who m= ay be interested.</span></p> <p style=3D"text-align: left; text-indent: 0px; text-transform: none; backg= round-color: rgb(255, 255, 255);"> <span style=3D"font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt= ; color: rgb(34, 34, 34);">Best regards,<br> Bhavya<br> Research Scientist, IBM</span></p> <p style=3D"text-align: left; text-indent: 0px; text-transform: none; backg= round-color: rgb(255, 255, 255);"> <span style=3D"font-family: Arial, Helvetica, sans-serif; font-size: 9.75pt= ; color: rgb(34, 34, 34);">Yuri Noviello<br> PhD Student, Delft University of Technology</span></p> <div style=3D"direction: ltr; font-family: "&quot;Aptos&quot;&= quot;, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"= > <br> </div> <div style=3D"direction: ltr; font-family: Aptos, Arial, Helvetica, sans-se= rif; font-size: 12pt; color: rgb(0, 0, 0);"> <br> </div> </body> </html> --_000_SA1PR15MB496778BC0DD7CD8D2917F1E5D6F12SA1PR15MB4967namp_-- --===============5557789845951762466== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Corpora mailing list -- [email protected] https://list.elra.info/mailman3/postorius/lists/corpora.list.elra.info/ To unsubscribe send an email to [email protected] --===============5557789845951762466==--