UCSC NLP at SemEval-2026 Task 10: Boundary-Aware Span Extraction and RoBERTa Classification for Conspiracy Detection
2026
SemEval-2026 Task 10 (PsyCoMark): conspiracy-marker span extraction and document-level detection
Subtask 1: Conspiracy Marker Extraction
- Built a RoBERTa-large span classifier over enumerated candidate spans (up to 32 tokens), with boundary-aware representations combining start/end embeddings, mean-pooled span content, span-width embeddings, and adjacent context tokens.
- Used IoU≥0.95 positive labeling with hard-negative mining (IoU 0.50–0.75) and per-role positive-class weighting to sharpen boundary discrimination under severe span imbalance.
- Designed containment-based non-maximum suppression and span merging to resolve overlapping and nested predictions across five conspiracy roles (Actor, Action, Effect, Evidence, Victim).
- Ranked 7th of participating systems on the official test set (0.2251 macro F1, token-level IoU≥0.5).
Subtask 2: Document-level Conspiracy Classification
- Fine-tuned a RoBERTa-large sequence classifier over the [CLS] representation to predict document-level conspiracy stance (Yes / No / Can't tell), trained independently from the span model with no cross-task feature sharing.
- Applied label smoothing and a stratified 90/10 train–validation split to stabilize training under class imbalance.
- Ranked 12th of participating systems on the official test set (0.7694 weighted F1).
