UnitedHealth’s Optum AI Chatbot Found Exposed Online, Access Restricted

Healthcare giant Optum, a subsidiary of UnitedHealth Group, restricted access to an internal AI chatbot after a cybersecurity researcher discovered it was inadvertently exposed to the internet, allowing public access without authentication.

The chatbot, called “SOP Chatbot,” was designed to assist employees by answering questions about handling health insurance claims and disputes based on the company’s standard operating procedures (SOPs). Although the bot did not process or store sensitive personal or health information, its exposure raises concerns given UnitedHealth Group’s ongoing scrutiny over its use of AI tools.

Security Flaw Discovered

Mossab Hussein, chief security officer at cybersecurity firm spiderSilk, found the chatbot accessible via its IP address despite being hosted on an internal Optum domain. The chatbot required no password for access.

It is unclear how long the chatbot was publicly accessible, but Optum removed access shortly after being contacted by TechCrunch.

An Optum spokesperson described the chatbot as a “proof-of-concept demo tool” and emphasized it was never in production or used in real-world applications. “This tool does not and would never make any decisions, but only enables better access to existing SOPs,” the company said in a statement.

Chatbot Functionality and Risks

The chatbot was trained on Optum’s internal SOP documents, which are stored on UnitedHealthcare’s corporate network and are normally restricted to employees. It assisted employees with tasks like determining claim eligibility and dispute resolution processes. However, the chatbot stored a history of employee interactions, including questions such as “What should be the determination of the claim?” and “How do I check policy renewal dates?”

A TechCrunch review found that the chatbot referenced internal documents and outlined common reasons for claim denials. Employees also used the chatbot for unrelated queries, such as jokes or attempts to “jailbreak” its functionality, with limited success.

Notably, when asked to “write a poem about denying a claim,” the chatbot produced a whimsical response that included lines such as:

“In the realm of healthcare’s grand domain,
Where policies and rules often constrain,
A claim arrives, seeking its due,
But alas, its fate is to bid adieu.”

Broader Implications

UnitedHealth Group, the parent company of Optum, has faced criticism and legal challenges over its use of artificial intelligence to allegedly deny patient claims. A federal lawsuit earlier this year accused the insurer of deploying AI tools with high error rates to improperly reject elderly patients’ care.

The company, the largest private healthcare insurer in the U.S., has denied the allegations and vowed to defend itself in court. UnitedHealth Group reported $22 billion in profits on $371 billion in revenue in 2023, underscoring its significant role in the healthcare industry.

The discovery of the exposed chatbot comes amid heightened scrutiny of AI applications in healthcare, particularly those that directly impact patient outcomes and decision-making processes. While Optum has assured that the chatbot never contained sensitive information, the incident highlights the need for stronger safeguards in deploying AI tools, even in internal testing environments.