FairStress: A deeper evaluation of fairness in LLM decision-making
huggingface ·
Allen AI has introduced FairStress, a new benchmark designed to evaluate demographic fairness in large language models beyond simple output choices. The study reveals that while many models select the correct candidate when demographics are swapped, they exhibit asymmetric robustness when challenged by persuasive adversarial prefixes. Specifically, models are more likely to abandon evidence-based decisions in favor of minority-coded candidates when pressured by arguments citing historical disadvantage or social consensus, a pattern described as anti-stereotypical over-correction. This asymmetry persists even when demographic identity is implied rather than explicitly stated, and it extends to differences in how models explain their reasoning. The research highlights that existing benchmarks like BBQ may miss these nuances because they rely on fixed-option scoring rather than generated responses. Furthermore, audits of training data suggest that unequal framing in fine-tuning corpora might contribute to these behavioral gaps. Although most debiasing interventions failed to fully resolve the issue, counterfactual chain-of-thought showed promise in reducing the gap without significantly sacrificing accuracy.
huggingface