LLMs From Major Tech Companies Lagging Behind in Safety Features, Says Benchmark Report

Recent findings from the Potential Harm Assessment & Risk Evaluation (PHARE) benchmark report reveal that large language models (LLMs) from major tech companies, such as OpenAI and Google, continue to fall short in safety and cybersecurity despite their financial growth for developers. Although Anthropic's models performed better, many LLMs, including some high-profile ones, display significant vulnerabilities to jailbreaks—a serious concern given the potential for manipulation and misinformation. The report indicates that the ability to resist attacks does not correlate with model size, highlighting an urgent need for healthcare technology professionals to prioritize security measures in AI deployment to safeguard sensitive information and ensure trustworthy interactions. Addressing these vulnerabilities is crucial as the healthcare sector increasingly adopts LLMs for clinical and administrative applications.

Opens on Dark Reading