UnitedHealth uses AI model with 90% error rate to deny care, lawsuit alleges - Ars Technica
Excerpt
For the largest health insurer in the US, AI’s error rate is like a feature, not a bug.
For the largest health insurer in the US, AI’s error rate is like a feature, not a bug.
UnitedHealthcare (UHC) health insurance company signage is displayed on an office building in Phoenix on July 19, 2023. Credit: Getty | Patrick Fallon
UnitedHealthcare, the largest health insurance company in the US, is allegedly using a deeply flawed AI algorithm to override doctors’ judgments and wrongfully deny critical health coverage to elderly patients. This has resulted in patients being kicked out of rehabilitation programs and care facilities far too early, forcing them to drain their life savings to obtain needed care that should be covered under their government-funded Medicare Advantage Plan.
That’s all according to a lawsuit filed this week in the US District Court for the District of Minnesota. The lawsuit is brought by the estates of two deceased people who were denied health coverage by UnitedHealth. The suit also seeks class-action status for similarly situated people, of which there may be tens of thousands across the country.
The lawsuit lands alongside an investigation by Stat News that largely backs the lawsuit’s claims. The investigation’s findings stem from internal documents and communications the outlet obtained, as well as interviews with former employees of NaviHealth, the UnitedHealth subsidiary that developed the AI algorithm called nH Predict.
“By the end of my time at NaviHealth I realized: I’m not an advocate, I’m just a moneymaker for this company,” Amber Lynch, an occupational therapist and former NaviHealth case manager, told Stat. “It’s all about money and data points,” she added. ‘It takes the dignity out of the patient, and I hated that.”
AI-based denials
According to the lawsuit, UnitedHealth started using nH Predict in at least November 2019, and it is still in use. The algorithm estimates how much post-acute care a patient on a Medicare Advantage Plan will need after an acute injury, illness, or event, like a fall or a stroke. Post-acute care can include things like therapy and skilled care from home health agencies, skilled nursing homes, and inpatient rehabilitation centers.
It’s unclear how nH Predict works exactly, but it reportedly estimates post-acute care by pulling information from a database containing medical cases from 6 million patients. NaviHealth case managers plug in certain information about a given patient—including age, living situation, and physical functions—and the AI algorithm spits out estimates based on similar patients in the database. The algorithm estimates medical needs, length of stay, and discharge date.
But Lynch noted to Stat that the algorithm doesn’t account for many relevant factors in a patient’s health and recovery time, including comorbidities and things that occur during stays, like if they develop pneumonia while in the hospital or catch COVID-19 in a nursing home.
According to the Stat investigation and the lawsuit, the estimates are often draconian. For instance, on a Medicare Advantage Plan, patients who stay in a hospital for three days are typically entitled to up to 100 days of covered care in a nursing home. But with nH Predict, patients rarely stay in nursing homes for more than 14 days before receiving payment denials from UnitedHealth.
When patients or their doctors have requested to see nH Predict’s reports, UnitedHealth has denied their requests, telling them the information is proprietary, according to the lawsuit. And, when prescribing physicians disagree with UnitedHealth’s determination of how much post-acute care their patients need, their judgments are overridden.
Favorable failings
The use of faulty AI is not new for the health care industry. While AI chatbots and image generators are currently grabbing headlines and causing alarm, the health care industry in the US has a longer record of problematic AI use, including establishing algorithmic racial bias in patient care. But, what sets this situation apart is that the dubious estimates nH Predict spits out seem to be a feature, not a bug, for UnitedHealth.
Since UnitedHealth acquired NaviHealth in 2020, former employees told Stat that the company’s focus shifted from patient advocacy to performance metrics and keeping post-acute care as short and lean as possible. Various statements by UnitedHealth executives echoed this shift, Stat noted. In particular, the UnitedHealth executive overseeing NaviHealth, Patrick Conway, was quoted in a company podcast saying: “If [people] go to a nursing home, how do we get them out as soon as possible?”
The lawsuit argues that UnitedHealth should have been well aware of the “blatant inaccuracy” of nH Predict’s estimates based on its error rate. Though few patients appeal coverage denials generally, when UnitedHealth members appeal denials based on nH Predict estimates—through internal appeals processes or through the federal Administrative Law Judge proceedings—over 90 percent of the denials are reversed, the lawsuit claims. This makes it obvious that the algorithm is wrongly denying coverage, it argues.
But, instead of changing course, over the last two years, NaviHealth employees have been told to hew closer and closer to the algorithm’s predictions. In 2022, case managers were told to keep patients’ stays in nursing homes to within 3 percent of the days projected by the algorithm, according to documents obtained by Stat. In 2023, the target was narrowed to 1 percent.
And these aren’t just recommendations for NaviHealth case managers—they’re requirements. Case managers who fall outside the length-of-stay target face discipline or firing. Lynch, for instance, told Stat she was fired for not making the length-of-stay target, as well as falling behind on filing documentation for her daily caseloads.
In an emailed statement, UnitedHealth’s subsidiary Optum Health told Ars:
The naviHealth predict tool is not used to make coverage determinations. The tool is used as a guide to help us inform providers, families and other caregivers about what sort of assistance and care the patient may need both in the facility and after returning home. Coverage decisions are based on CMS coverage criteria and the terms of the member’s plan. This lawsuit has no merit, and we will defend ourselves vigorously.
Ultimately, case managers do not decide on coverage or denials—those decisions fall to NaviHealth’s physician medical reviewers. But, those physicians are advised by the case managers, who are held to the 1 percent target.
And case managers are specifically trained to defend the algorithm’s estimate to patients and their care providers. One training document obtained by Stat discussed the blunt tactics case managers were told to take when patients and caregivers pushed back on denials. It stated:
- If a nursing home balked at discharging a patient with a feeding tube, case managers should point out that the tube needed to provide “26 percent of daily calorie requirements” to be considered as a skilled service under Medicare coverage rules.
- If a nurse took a broader tack, and argued a patient was unsafe to leave, case managers were instructed to counter, in part, that the algorithm’s projections about a patient’s care needs, and readiness for discharge, are based on a “severity-adjusted” comparison to similar patients around the country. “Why would this patient be any different?” the document asks.
No winning
Even for the patients who appeal their AI-backed denials and succeed at getting them overturned, the win is short-lived—UnitedHealth will send new denials soon after, sometimes within days.
A former unnamed case manager told Stat that a supervisor directed her to immediately restart a case review process for any patient who won an appeal. “And 99.9 percent of the time, we’re going to turn right back around and issue another [denial],” the former case manager said. “Well, you won, but OK, what’d that get you? Three or four days? You’re going to get another [denial] on your next review, because they want you out.”
The plaintiffs leading the proposed class-action suit include the family of Gene Lokken, who died on July 17 of this year. On May 5 2022, the 91-year-old fell at home, fracturing his leg and ankle. After around six days in the hospital, he was moved to hospice care, where he spent a month recovering from his injuries. After that, doctors said he became well enough to start physical therapy. But UnitedHealth only paid for 19 days of therapy, dumbfounding his doctors and therapists, who described his muscle functions as “paralyzed and weak.” The family appealed the denial, but their appeal was rejected. The rejection letter UnitedHealth sent the family said additional physical therapy was unneeded because there were no acute medical issues, and he was self-feeding and required minimal help for hygiene and grooming.
The family had no choice but to pay out of pocket for his therapy, spending around $150,000 until his death.
The other plaintiff is the family of Dale Tetzloff, who suffered a stroke on October 4, 2022, and was admitted to a hospital. While there, the 74-year-old’s doctors referred him to a skilled nursing home and determined he would need at least 100 days of post-acute care. But, after 20 days at the skilled nursing home, UnitedHealth denied further coverage.
His family appealed the denial, twice, overturning it on the second appeal after NaviHealth doctors reviewed Tetzloff’s medical records. But, after 40 days at the skilled nursing home, UnitedHealth denied coverage again and refused to provide a reason. The family continued trying to appeal the denial, but were unsuccessful. Meanwhile, they paid $70,000 out of pocket over about 10 months. In June, 2023, he was moved to an assisted living facility, where he died on October 11.
The lawsuit accuses UnitedHealth and NaviHealth of breach of contract, breach of good faith and fair dealing, unjust enrichment, and insurance law violations in many states. It calls for actual damages, damages from emotional distress, disgorgement and/or resititution, and an end to the AI-based claims denials.
It’s unclear how much UnitedHealth saves by using nH Predict, but Stat estimated it to be hundreds of millions of dollars annually. In 2022, UnitedHealth Group’s CEO made 10 and $16 million each.
Beth is Ars Technica’s Senior Health Reporter. Beth has a Ph.D. in microbiology from the University of North Carolina at Chapel Hill and attended the Science Communication program at the University of California, Santa Cruz. She specializes in covering infectious diseases, public health, and microbes.
