Sarc7: Evaluating Sarcasm Detection and Generation with Seven Types and Emotion-Informed Techniques

Researchers introduce Sarc7, a benchmark dataset for classifying seven types of sarcasm using large language models, with a novel emotion-based prompting technique that outperforms traditional zero-shot and few-shot approaches. The study demonstrates that Gemini 2.5 achieved the highest performance with an F1 score of 0.3664, while emotion-informed generation methods showed 38.46% improvement in human evaluation over baseline approaches.

The Sarc7 benchmark addresses a fundamental challenge in natural language processing: accurately detecting and generating sarcasm, a nuanced form of humor that requires understanding contextual incongruity and implied emotional undertones. Sarcasm detection has long been difficult for computational models because it often requires knowledge of cultural context, speaker intent, and subtle linguistic cues that literal semantic analysis cannot capture. This research tackles that gap by proposing emotion-based prompting techniques that explicitly incorporate emotional dimensions into the model's reasoning process.

The research builds on existing sarcasm datasets like MUStARD but advances the field by categorizing sarcasm into seven distinct types rather than treating it as a binary classification problem. This granularity is important because different sarcasm types carry different emotional signatures and contextual markers. The emotion-based prompting methodology represents a shift toward more sophisticated prompt engineering, moving beyond simple zero-shot or few-shot approaches to architecturally integrate emotional reasoning.

For the AI development community, these findings suggest that injecting emotional or semantic context into prompts can significantly improve model performance on nuanced language tasks. The 38.46% improvement in human-preferred generations indicates practical value for applications like chatbots, content moderation systems, and sentiment analysis tools. The research also demonstrates that Gemini 2.5's architecture effectively processes emotional context when properly prompted, informing decisions about which models to use for complex language understanding tasks.

Future work should explore whether emotion-based techniques transfer to other figurative language tasks like irony or metaphor detection, and whether these methods improve real-world deployment scenarios beyond benchmark evaluation.

• →Emotion-based prompting achieved 0.3664 F1 score with Gemini 2.5, outperforming traditional zero-shot and few-shot methods for sarcasm classification.
• →Sarc7 categorizes sarcasm into seven distinct types—self-deprecating, brooding, deadpan, polite, obnoxious, raging, and manic—providing granular classification beyond binary approaches.
• →Emotion-informed generation methods produced 38.46% more successful outputs compared to zero-shot prompting according to human evaluation.
• →The research identifies incongruity, shock value, and context dependency as key components of effective sarcasm generation.
• →Results suggest prompt engineering incorporating emotional context significantly improves LLM performance on nuanced language understanding tasks.