PAN at CLEF 2021

While models built and tested on a specific dataset and for a specific task often achieve very good performance, they then fail to generalise when they are applied to new, unseen data. In this talk I will discuss the importance and challenges of achieving generalisable performance in social media research with a particular focus on fact verification and hate speech detection. I will present some of our recent work in this direction, as well as discuss open challenges to further the capacity of generalisation especially in hate speech detection.

Language has the power to reinforce stereotypes and project social biases onto others, either through overt hate or subtle biases. Accounting for this toxicity and social bias in language is crucial for natural language processing (NLP) systems to be safely and ethically deployed in the world. In this talk, I will first analyze a failure case of automatic hate speech detection, in which we find that models tend to flag speech by African Americans as toxic more often than by others. We trace the origins of the biases back to the annotated datasets, and show that we can reduce these biases, by making a tweet's dialect more explicit during the annotation process. Then, as an alternative to binary hate speech detection, I will present Social Bias Frames, a new structured formalism for distilling biased implications of language. Using a new corpus of 150k structured annotations, we show that models can learn to reason about high-level offensiveness of statements, but struggle to explain why a statement might be harmful. Finally, I will introduce PowerTransformer, a new unsupervised model for controllable debiasing of text through the lens of connotation frames of power and agency. With this model, we show that subtle gender biases in how characters are portrayed in stories and movies can be mitigated through automatic rewriting. I will conclude with future directions for better reasoning about toxicity and social biases in language.

Building emotional connection is key for human beings, but sometimes in the business world we forget about it, we forget that behind every potential deal there is a person, and we just speak about leads, customers or churn... At Symanto, we combine Artificial Intelligence with Psychology to help our customers build emotional connections at scale to improve the way they resonate and communicate. In this Industry talk, I will briefly introduce Symanto, how we apply, among others, author profiling techniques, how our core technology allows us to stand out from our competition, why emotions and psychology matter, how we offer this to the world, as well as some of the latest research lines that we are working on.

Multi-authorship tasks have been run at PAN now for over half a decade. In this time, several multi-authorship tasks have been identified, a number of datasets have been created, and numerous methods have been developed to address the tasks. The community here has achieved considerable success, given these challenging tasks. To use an analogy, the multi-authorship jigsaw puzzle is now starting to take place, with many of the most important foundational pieces in place. In this talk, I will speculate on the next possible pieces of the multi-authorship jigsaw to connect the larger pieces of the puzzle, focusing on the construction of datasets and evaluation of methods.