Emotion Detection

Emotion Detection aims to classify a fine-grained emotion for each utterance in multiparty dialogue. Our annotation is based on the primary emotions in the Feeling Wheel (Willcox, 1982). We must admit that the inter-annotator agreement of this annotation is not the greatest; we welcome any contribution from the community to improve the annotation quality. This task is a part of the Character Mining project led by the Emory NLP research group.

Dataset

Each utterance is annotated with one of the seven emotions, sad, mad, scared, powerful, peaceful, joyful, and neutral, that are the primary emotions in the Feeling Wheel.

• Latest release: v1.0.
• Release notes.

Statistics

The following episodes are used for the training, development, and evaluation sets:

• Train (TRN): [s01_e02, s01_e03, s01_e04, s01_e05, s01_e06, s01_e07, s01_e08, s01_e09, s01_e11, s01_e12, s01_e13, s01_e14, s01_e16, s01_e17, s01_e18, s01_e19, s01_e21, s01_e22, s01_e23, s01_e24, s02_e01, s02_e02, s02_e03, s02_e04, s02_e05, s02_e06, s02_e07, s02_e09, s02_e11, s02_e12, s02_e13, s02_e14, s02_e15, s02_e16, s02_e17, s02_e18, s02_e19, s02_e21, s02_e22, s02_e24, s03_e02, s03_e03, s03_e04, s03_e05, s03_e06, s03_e07, s03_e10, s03_e11, s03_e12, s03_e13, s03_e14, s03_e15, s03_e16, s03_e17, s03_e18, s03_e19, s03_e22, s03_e23, s03_e24, s03_e25, s04_e03, s04_e04, s04_e05, s04_e07, s04_e08, s04_e09, s04_e11, s04_e12, s04_e13, s04_e14, s04_e15, s04_e16, s04_e18, s04_e19, s04_e22, s04_e23, s04_e24]
• Development (DEV): [s01_e15, s01_e20, s02_e10, s02_e20, s03_e01, s03_e09, s03_e21, s04_e01, s04_e06, s04_e10, s04_e21]
• Evaluation (TST): [s01_e01, s01_e10, s02_e08, s02_e23, s03_e08, s03_e20, s04_e02, s04_e17, s04_e20]

Dataset	Episodes	Scenes	Utterances
TRN	77	713	9,934
DEV	11	99	1,344
TST	9	85	1,328
Total	97	897	12,606

Dataset	Neutral	Joyful	Peaceful	Powerful	Scared	Mad	Sad	Total
TRN	3,034	2,184	900	784	1,285	1,076	671	9,934
DEV	393	289	132	134	178	143	75	1,344
TST	349	282	159	145	182	113	98	1,328
Total	3,776	2,755	1,191	1,063	1,645	1,332	844	12,606

Annotation

Each utterance has the field emotion. Three utterances in the following example are annotated with the emotions of Neutral, Joyful, and Powerful, respectively.

{
  "utterance_id": "s01_e02_c01_u002",
  "speakers": ["Joey Tribbiani"],
  "transcript": "Yeah, right!.......Y'serious?",
  "tokens": [
    ["Yeah", ",", "right", "!"],
    ["......."],
    ["Y'serious", "?"]
  ],
  "emotion": "Neutral"
},
{
  "utterance_id": "s01_e02_c01_u003",
  "speakers": ["Phoebe Buffay"],
  "transcript": "Oh, yeah!",
  "tokens": [
    ["Oh", ",", "yeah", "!"]
  ],
  "emotion": "Joyful"
},
{
  "utterance_id": "s01_e02_c01_u004",
  "speakers": ["Rachel Green"],
  "transcript": "Everything you need to know is in that first kiss.",
  "tokens": [
    ["Everything", "you", "need", "to", "know", "is", "in", "that", "first", "kiss", "."]
  ],
  "emotion": "Powerful"
}

Citation

• Emotion Detection on TV Show Transcripts with Sequence-based Convolutional Neural Networks. Sayyed Zahiri and Jinho D. Choi. In The AAAI Workshop on Affective Content Analysis, AFFCON'18, 2018.

Contact

• Jinho D. Choi.