A group of state attorneys general have issued a warning to major AI companies, including Microsoft, OpenAI, and Google, urging them to address ‘delusional outputs’ generated by AI systems to protect users from potential psychological harm. The letter, signed by numerous AGs and backed by the National Association of Attorneys General, highlights concerns over recent mental health incidents involving AI chatbots.
The AGs are pushing for the implementation of new internal safeguards such as transparent third-party audits of AI models to identify delusional or sycophantic ideations. Additionally, they recommend the adoption of incident reporting procedures to alert users when AI systems produce psychologically harmful content. The letter emphasizes the need for independent third parties to evaluate systems pre-release without facing repercussions and to publish their findings without company approval.
The warning underscores the dual nature of AI, acknowledging its potential to positively impact society while also recognizing the risks it poses, particularly to vulnerable populations. The AGs draw attention to instances where AI-generated outputs have been linked to tragic events, prompting a call for enhanced oversight and transparency in handling mental health incidents related to AI usage.
By likening the response to mental health incidents to cybersecurity incident protocols, the AGs aim to ensure a more structured and accountable approach from tech companies in addressing potential AI-related psychological harm.
Source: TechCrunch
Leave a Reply