Biased AI in health care faces crackdown in sweeping Biden admin proposals

Prior authorization

Elsewhere in the over 700-page proposal, the administration lays out policy that would bar Medicare Advantage plan providers from reopening and reneging on paying claims for inpatient hospital admission if those claims had already been granted approval through prior authorization. The proposal also wants to make criteria for coverage clearer and help ensure that patients know they can appeal denied claims.

The Department of Health and Human Services notes that when patients appeal claim denials from Medicare Advantage plans, the appeals are successful 80 percent of the time. But, only 4 percent of claim denials are appealed—”meaning many more denials could potentially be overturned by the plan if they were appealed.”

AI guardrails

Last, the administration’s proposal also tries to shore up guardrails for the use of AI in health care with edits to existing policy. The goal is to make sure Medicare Advantage insurers don’t adopt flawed AI recommendations that deepen bias and discrimination or exacerbate existing inequities.

As an example, the administration pointed to the use of AI to predict which patients would miss medical appointments—and then recommend that providers double-book the appointment slots for those patients. In this case, low-income patients are more likely to miss appointments, because they may struggle with transportation, childcare, and work schedules. “As a result of using this data within the AI tool, providers double-booked lower-income patients, causing longer wait times for lower-income patients and perpetuating the cycle of additional missed appointments for vulnerable patients.” As such, it should be barred, the administration says.

In general, people of color and people of lower socioeconomic status tend to be more likely to have gaps and flaws in their electronic health records. So, when AI is trained on large data sets of health records, it can generate flawed recommendations based on that spotty and incorrect information, thereby amplifying bias.

Leave a Comment

Your email address will not be published. Required fields are marked *