A study by the Anti-Defamation League (ADL) has revealed significant disparities in the performance of AI chatbots in identifying and addressing antisemitic content. The evaluation, which included models from Anthropic, OpenAI, Meta, Google, and DeepSeek, found that while Anthropic’s Claude exhibited the strongest capabilities, all the tested chatbots displayed deficiencies that require further enhancement.
The ADL’s assessment encompassed various conversational scenarios, including responding to statements, providing evidence for and against claims, and generating talking points based on content related to antisemitism, anti-Zionism, and extremism. While Claude was commended for its performance, the study highlighted a 59-point gap between it and the lowest-ranked model, xAI’s Grok.
The rankings from best to worst were Claude, ChatGPT, DeepSeek, Gemini, Llama, and Grok. The ADL’s emphasis on the need for improvement across all models underscores the ongoing challenge of developing AI systems that can effectively address sensitive societal issues.
Source: The Verge