Register now!

Celebrity Profiling

PAN @ CLEF 2019

Given the English Social Media feed of a celebrity, determine their degree of fame, occupation, age, and gender.

Task

Celebrities are among the most prolific users of social media, promoting their personas and rallying followers. This activity is closely tied to genuine writing samples, rendering them worthy research subjects in many respects, not least author profiling.

The Celebrity Profiling task this year is to predict four traits of a celebrity from their social media communication. The traits are the degree of fame, occupation, age, and gender. The social media communication is given as the teaser messages from past tweets. The goal is to develop a piece of software which predicts celebrity traits from the teaser history.

Total Dataset Size 48,335 User Profiles
text size 2,181 Tweets avg. per User
novel traits Fame and Occupation
New Attributes Detailed Birthyears and Nonbinary Gender.

Data [Download Training Data]

The training dataset contains of two files: a feeds.ndjson as input and a labels.ndjson as output. Each file lists all celebrities as JSON objects, one per line and identified by the id key.

Input Format

The input file contains the cid and a list of all teaser messages for each celebrity.
{"id": 1234, "text": ["a tweet", "another tweet", ...]}
{"id": 5678, "text": ["a tweet", "another tweet", ...]}
...
feeds.ndjson

Output Format

The output file contains the cid and and a value for each trait for each celebrity from the input file.
{"id": 1234, "fame": "star", "occupation": "sports", "gender": "female", "birthyear": 2002}
{"id": 5678, "fame": "rising", "occupation": "professional", "gender": "male", "birthyear": 1990}
...
labels.ndjson
The following values are possible for each of the traits:
fame        := {rising, star, superstar}
occupation  := {sports, performer, creator, politics, manager,
                science, professional, religious}
birthyear   := {1940, ..., 2012}
gender      := {male, female, nonbinary}
possible value instances for each label

Evaluation [Evaluation Script][Unittests]

Submissions are judged by a combined metric cRank, which is the harmonic mean of each label's metric. $$ \text{cRank} = {4 \over {\frac{1}{\text{F}_{1, \text{fame}}} + \frac{1}{\text{F}_{1, \text{occupation}}} + \frac{1}{\text{F}_{1, \text{gender}}} + \frac{1}{\text{F}_{1, \text{age}}}}} $$ All traits are judged by their respective F1. Precision and recall of birthyear are calculated leniently. If a prediction is within an m-window of the truth, it is counted as correct: $$ \text{true birthyear} - m \le \text{predicted birthyear} \le \text{true birthyear} + m$$ The window size m is based on the birth year and increases linearly from about 2 years for 2012 to about 9 years for 1940.

Submission [Submit software to PAN]

For evaluation, your software will read a feeds.ndjson file from a given directory and write a valid labels.ndjson with your predictions to a given output directory.

This task follows PAN's software submission strategy described here.

Related Work


Data Analysis