Adding a “Human In the Loop” Made Claude Worse

Claude Opus 5 scored 100% on all 20 attempts at four month-end close tasks. Then researchers brought in licensed CPAs, and the results got worse. That’s the most cringe-inducing finding in a new study from Mercor, a startup that uses professionals to train and evaluate AI models.