Training set released
Registration Form:
https://forms.gle/yrpbJPqxCuXB7sN76.Mailing List:
https://groups.google.com/g/ediref2024_group.CodaLab Link:
https://codalab.lisn.upsaclay.fr/competitions/14567.The EDiReF shared task at SemEval 2024 is an amalgamation of three subtasks tasks- (i) Emotion Recognition in Conversation (ERC) in Hindi-English code-mixed conversations, (ii) Emotion Flip Reasoning (EFR) in Hindi-English code-mixed conversations, and (iii) EFR in English conversations. The definition for the tasks of ERC and EFR are as illustrated below:
- ERC: Given a dialogue, ERC aims to assign an emotion to each utterance from a predefined set of possible emotions.
- EFR: Given a dialogue, EFR aims to identify the trigger utterance(s) for an emotion-flip in a multi-party conversation dialogue.
For example, given a sample dialogue below:
| Speaker | Utterance |
|---|---|
| Sp1 | Aaj to bhot awful day tha! (I had an awful day today!) |
| Sp2 | Oh no! Kya hua? (Oh no! What happened?) |
| Sp1 | Kisi ne mera sandwich kha liya! (Somebody ate my sandwich!) |
| Sp2 | Me abhi tumhare liye new bana deti hun! (I can make you a new one right now!) |
| Sp1 | Wo great hoga! Thanks! (That would be great! Thanks!) |
ERC aims to assign emotions to each utterance
| Speaker | Utterance | Emotion |
|---|---|---|
| Sp1 | Aaj to bhot awful day tha! (I had an awful day today!) | Sad |
| Sp2 | Oh no! Kya hua? (Oh no! What happened?) | Sad |
| Sp1 | Kisi ne mera sandwich kha liya! (Somebody ate my sandwich!) | Sad |
| Sp2 | Me abhi tumhare liye new bana deti hun! (I can make you a new one right now!) | Joy |
| Sp1 | Wo great hoga! Thanks! (That would be great! Thanks!) | Joy |
The example dialogue has two emotion flips. Sp1's emotion changed from Sad to Joy while Sp2's emotion also shifted from Sad to joy. EFR aims to justify such emotion flips using triggers.
| Speaker | Utterance | Emotion | Trigger |
|---|---|---|---|
| Sp1 | Aaj to bhot awful day tha! (I had an awful day today!) | Sad | 0 |
| Sp2 | Oh no! Kya hua? (Oh no! What happened?) | Sad | 0 |
| Sp1 | Kisi ne mera sandwich kha liya! (Somebody ate my sandwich!) | Sad | 0 |
| Sp2 | Me abhi tumhare liye new bana deti hun! (I can make you a new one right now!) | Joy | 1 |
| Sp1 | Wo great hoga! Thanks! (That would be great! Thanks!) | Joy | 0 |
Important Dates
| Evaluation start | 10 January 2024 |
| Evaluation end | 20 January 2024 |
| Paper submission due | 29 February 2024 |
| Notification to authors | 1 April 2024 |
| Camera ready due | 22 April 2024 |
| SemEval workshop | TBD, 2024 (co-located with a Major NLP conference) |
Data
Kindly register to the Codalab competition to get started with model submission
CodaLab
CodaLab link for the shared task: https://codalab.lisn.upsaclay.fr/competitions/14567. Interested participants are encouraged to register now and join the mailing list to stay updated.
Organization
The organizers are:
- Shivani Kumar, IIIT, Delhi
- Md Shad Akhtar, IIIT, Delhi
- Tanmoy Chakraborty, IIT, Delhi
- Erik Cambria, NTU, Singapore