Things are relatively quiet on the AI front, so I figured it’s time to check in on some other things that have been going on, including various developments at the FDA.
Matthew Herper: FDA chief Marty Makary says ‘everything should be over the counter’ unless drug is unsafe or addictive [or requires monitoring].
Annika Kim Constantino: Makary said the FDA is looking at “basic, safe” prescription drugs like nausea medications and vaginal estrogen, which is used to treat menopausal symptoms like dryness and pain.
You’d also want a carveout for antibiotics, but yes. Your offer is acceptable, LFG.
That was the good news. There’s also the bad news, which was the main event.
Settle in for the tale of Moderna, and its attempt to get approval for a flu vaccine.
They were told their study design was acceptable. Then they got rug pulled. Then a lot of people were very loud and the FDA backtracked a bit while pretending not to.
Jeremiah Johnson: I remember reading that the EA case for Trump was that we’d see a flood of new drugs with a more innovation-friendly FDA.
Alex Tabarrok: I Regret to Inform You that the FDA is FDAing Again.
I had high hopes and low expectations that the FDA under the new administration would be less paternalistic and more open to medical freedom. Instead, what we are getting is paternalism with different preferences. In particular, the FDA now appears to have a bizarre anti-vaccine fixation, particularly of the mRNA variety (disappointing but not surprising given the leadership of RFK Jr.).
mattparlmer: I’ve been restraining myself from political commentary for the last few weeks, but I would be remiss not to register my extreme disappointment at the groundless sacking of Jim O’Neill
The Trump admin desperately needs competent people, Jim is exceptionally competent, this is bad
This is what happens when you burn the term supervillain on people who want lower prices on drugs. What do you then call people who outright destroy one of the most promising medical advances of our time for no benefit whatsoever, via torching a $750 million study for decisions the FDA had explicitly already approved.
In this case, via withholding review for a drug that they had no business requiring the study for in the first place, in order to retroactively demand an absurd standard?
The FDA claims no, they didn’t say this study design was valid, but they’re lying, of course they did, there’s no way you spend $750 million dollars here without checking.
CNN Breaking News: The FDA has refused to accept an application from Moderna to review its first mRNA seasonal flu vaccine, the company said.
The agency told Moderna that its application didn’t contain an “adequate and well-controlled” trial because the control arm didn’t reflect the “best-available standard of care in the United States at the time of the study,” according to a letter dated February 3 that Moderna posted online. It didn’t identify any safety or efficacy concerns, the company said.
Moderna said that the refusal was inconsistent with previous feedback from the agency and that it had requested a meeting with FDA officials to understand how to proceed.
Alex Tabarrok: Hilarious: Trump admin is torn between those who say Moderna’s flu trial was unethical because it didn’t use a high-dose vaccine as comparator, exposing participants to “increased risk of severe illness,” and those who say it was unethical because it didn’t use an inert placebo!
Per STAT, Prasad overruled FDA scientists to reject the application outright, and we have confirmation that staff was ready to review.
As an obvious technical note, we already have extensive data on how different existing vaccines relate to each other, so even if you thought that we shouldn’t allow a new offering until it was proven strictly superior and you changed your mind about the current standard, it is not statistically difficult to back out the necessary information.
Ruxandra Teslo: This is a very bad precedent. Regulatory agencies changing their stance after ok ing a study is one of the worst things for innovation.
Sam D’Amico: This is going to waste a week of my time every other year for the next 20 years
Leah Libresco Sargeant: “What is most disturbing is not the specifics of this case but the arbitrariness and capriciousness of the process. The EU, Canada, and Australia have all accepted Moderna’s application for review.”
Christina Jewett (NYT, February 11): UPDATE — FDA defends this refusal, but also retreats, calling this routine dialog with drug makers and saying it will work with Moderna —
The Food and Drug Administration on Wednesday backtracked on its refusal to review Moderna’s mRNA flu vaccine, while making an unusual public defense of its decisions on drug approvals.
… Dr. Marty Makary, the F.D.A. commissioner, suggested during a Fox News interview on Wednesday that the agency might eventually approve the vaccine. He characterized the rejection of Moderna’s application as “part of a conversation that occurs with all drug makers” and said the agency would continue to work with the company.
Matthew Herper: Here the Moderna flu data the FDA refused to review.
This is the most important slide from the company’s study of its experimental flu vaccine– exactly the data FDA said it wouldn’t even review.
And below is a thread of relevant slides from that presentation and links to other relevant studies I wanted to bookmark. (I’m a reporter who covers this stuff.)
Matthew Herper: As you can see, volunteers who received Moderna’s flu shot, mRNA-1010, had about 26% less influenza-like illness in a 40,000 patient study.
These slides were presented to Moderna investors in November. Vinay Prasad, a top FDA official, told the company the vaccine would not be reviewed because the company compared to a standard, not high-dose, flu vaccine, and high-dose vaccines are standard for people over 65.
Here’s more background on the study. It enrolled, 40,000 people. They’re about evenly split between the 50-64 age group and the 65+ group.
mRNA vaccines can cause fatigue, achiness, and headache. They certainly saw that here.
Here’s the efficacy for different strains of influenza.
The full slides are available here. It is not obvious this particular vaccine is a good deal for most people, but it clearly works, and they did what the FDA said to do. The FDA pulled the plug anyway, not even examining the findings.
It is hard to overstate how destructive this is. This meaningfully hurts your quality of life, the number of days per year you can expect to be sick, and your long term life expectancy.
That’s because if Moderna has to worry about the FDA retroactively withdrawing its word on what studies will be accepted, after the work is done, and they are facing an mRNA-hostile FDA in general, they have no choice. They were already abandoning much of the work.
And everyone else has to have the same worry everywhere else. Oh no. At minimum, this forces ballooning costs as studies have to be designed defensively in case the wrong person at the FDA suddenly changes their mind.
Good luck implementing risk-based monitoring (RBM) or otherwise cutting costs, and many potential drugs will be abandoned.
unusual_whales: Moderna’s, $MRNA, CEO has said that the company will no longer invest in new Phase 3 vaccine trials for infectious diseases: ‘You cannot make a return on investment if you don’t have access to the U.S. market.’ Vaccines for Epstein-Barr virus, herpes, and shingles have been shelved.
Dr. Lakshay Mittal: When late-stage programs for Epstein–Barr, herpes and shingles get shelved, it shifts the pipeline risk profile of the entire biotech sector.
Investors will start asking which other infectious disease programs depend heavily on US market access.
Nick Mark MD: Three points: Moderna and FDA had previously agreed on the control arm dose. FDA went back on that agreement.
Presumably Prasad signed it because everyone else refused to.
If FDA pulls a switch like this it really kills biotechs incentive to do big trials. (this was n=40k)
The pivot by Moderna to not invest in new Phase 3 trials was announced at Davos in January in the face of other symptoms of FDA hostility. Every person who gets Epstein-Barr, herpes or shingles more than a few years from now should know exactly who to blame for this outcome. That is only the start. Other companies, even those not focused on mRNA, have to now factor in this risk.
The good news is that we managed to yell loudly enough about this one that they may refuse course.
Christina Jewett: Two points here. First, one of these two people is lying, and it would be a hell of a thing if the liar was Moderna. Second, whether people were previously from some point of view exposed to risk of illness has zero bearing on whether you should now expose everyone to risk of illness.
Saying in hindsight that the trial you approved was ‘unethical’ when it was clearly net beneficial to patients and there’s no case this contaminated the study? Complete and utter bullshit.
Especially when you’re simultaneously arguing the exact opposite:
Mr. Kennedy long raised concerns as a vaccine activist that the inactive ingredients in vaccines could cause harm and should not be used in human trials.
They say Kennedy was not involved here, but he’s head of the agency and has been furiously acting to undercut Moderna and its mRNA research at every turn.
One can try to frame this all as business as usual, and there are advantages to everyone coming away thinking about it that way whether or not it’s true:
Jessica Adams: The bad news is that I do not believe this was remotely normal. A lot of the damage is done and cannot be undone. The procedural decision itself is bad enough even if it is ultimately reversed. Mock the ‘chilling effect’ at your peril.
Venk Murthy tries to frame this as ‘rebuilding trust’ by the public, but I expect this only further inflames popular paranoia about vaccines, because of the reversal, and the key ‘rebuilding trust’ that is suddenly necessary is the trust in the FDA by those researching vaccines. Without that, no new vaccines.
Alex Tabarrok shares this, which summarizes the ‘medical freedom’ movement driving much of this. Do you really think you can ‘rebuild trust’ with such folks?
It is important, when people’s objections are Obvious Nonsense, to point this out.
Stanford researchers identified 16-PGDH as a root cause of cartilage loss, claiming the mechanism works IN MICE, and also in human tissue samples. Epirium for now is doing Phase 1 trials for sarcopenia. FDA Delenda Est so we’re looking even in an optimistic case at 5-7 year timelines, but if it all works it’s possible this solves not only arthritis but a wide range of quality of life issues. Alas, this won’t do that much for life expectancy on its own since these mostly aren’t the things that kill you, but it could be a big win in QALY terms if it works out.
How should you describe your situation to a doctor? You want to be respectful and honest with them, but if you dump a bunch of information they can get easily distracted in predictable ways. I buy Carl Feynman’s advice here to start with your diagnosis and key facts that seem most relevant and unlikely to mislead, then only later give the rest. Think of it as (polite) prompt engineering. LLMs can help you.
This is one way to do plastic surgery, but you’d better be damn sure you’ve solved the associated principal-agent problems and the surgeon has good taste:
Aella: Y’all my plastic surgery made me look moderately cuter. The feeling of seeing yourself more attractive in the mirror for the first time after the swelling finally fades is transcendent. You’ll have to pry the scalpel from my cold dead hands.
Aella: my strategy this time was to go into the surgeon’s office, refuse to tell him my own concerns, ask “what would you do to make me look hotter”, and then blanket accept everything he suggested – even if I hadn’t thought I needed it.
It seems right that most people are not good judges of what would or would not make them look hotter, and that experts will know better, but also those experts have incentives. Perhaps you should focus more on establishing trust than trying to solve the problem yourself.
Also the person who most has to look at you will be you, so your preferences matter. Don’t fully ignore them. And yes, there is cumulative risk of backfire if you overdo it.
AI but not LLMs: New large study (n = 105,934) of AI in cancer screening that started in April 2021 finds strong results from Transpara Version 1.7.0. Gains plausibly were largely about workflow integration and use of risk scores, rather than AI as a standalone second reader. Descriptively, there were overall 16% fewer invasive, 21% fewer large (T2+), and 27% fewer non-luminal A interval cancers in the intervention group compared with the control group, and the detection rate of cancer rose 29%, while enabling a 44% reduction in radiologist ‘screen readings.’ They got increased sensitivity without loss of specificity.
I am deeply confused by the whole ‘we still don’t know if this would be worth the cost’ response. What cost?
Mohamed Foda: 100K women. 29% more cancers caught. 44% less workload. And somewhere a hospital administrator is still asking for a 3-year committee review before adopting AI. This is what happens when medicine actually evolves.
That’s for old school AI that was available back in 2021. Diffusion not going great.
Steven Pinker: The autism epidemic is a myth: The increase is only in expansive overdiagnosis of kids “on the spectrum,” e.g. “Would rather be alone than with others,” “Has difficulty making friends,” and “Is regarded by other children as odd or weird.” Severe cases (no language, socially unresponsive, etc.) have decreased. By my former grad student @AdamOmaryPhD
Justin Gregory Briggs: In the 20+ years I’ve been in the mental health field, the diagnostic criteria for autism have become so broad that they are clinically useless. Any spectrum that includes some of the most productive people on Earth and people who require 24/7 care is not interpretable.
Universal mental health screening is ineffective at creating prosocial behavior and improved academic outcomes. The practice is also harmful – and occurs almost wholly unregulated by state boards of behavioral health.
Carolyn D. Gorman: This issue brief provides an overview of universal school-based mental health screening, related law and evidence, and practical and ethical concerns. It finds that:
Universal mental health screenings are ineffective: they do not reduce the prevalence of mental health conditions or improve academic outcomes.
Universal mental health screenings are harmful: they produce overwhelmingly high rates of false positives and, when delivered in schools, have fewer protections in place than in clinical settings.
Given the lack of evidence for universal mental health screening, this brief recommends its prohibition in schools. For targeted and individual school-based mental health screenings, far stronger regulatory conditions are warranted. Model legislation is provided for implementing these recommendations.
Mental health screenings sound like a good idea, but the false positives and overtreatment and medicalization of problems that result in practice mean they end up doing more harm than good. Such screenings look for trouble, and trouble they often find, which can be devastating for kids and families.
Adam: There are many examples of waste and inefficiency in clinical trials, but I want to highlight among these, one practice stands out: a practice called 100% source data verification, which is both distinctly wasteful and can tell us a lot about what does – and what doesn’t – drive the inefficiencies in today’s trials.
… After the site sends the data they collected to the sponsor, the sponsor sends a consultant to the site to pore through their paper records to make sure that every single data element the site collected matches the data the sponsor received.
That seems rather obviously crazy. Even the FDA now agrees. Yet Adam claims this is what typically happens.
100% source data verification is a huge task. The typical phase 3 clinical trial collects 3 million data points. So it is perhaps not surprising that it’s expensive to check them all: the process of 100% source data verification accounts for 25%-40% of clinical trial costs. And it’s completely pointless. It doesn’t meaningfully help with clinical trial quality. The FDA has publicly and repeatedly recommended against doing it for over ten years.
Nobody has a very good explanation for why the industry continues this practice.
Easy. Ban the practice, as in have the FDA require you not to do it. Alternatively, Claude thinks the concrete fear is how things will look on audits, so you can create rules to protect you from audits.
Instead we have the opposite, now more than ever. The FDA is announcing it will look for any excuse to deny your application in at least some cases.
The question then is, are there cases where the FDA will look for any excuse to say no, but end up having to say yes anyway? There might not be. A common pattern when seeking approval is that you beat yourself up over details that might matter, and then you get told one of your choices mattered, when really their mind was already made up for other reasons.
Also I saw claims that this particular data point does not hold up. Claude notes SPV is not universal, but thinks the claim of 25%-40% of total costs is plausible.
Unfortunately, today’s approach to clinical trials looks much more like NASA’s “Big Space” approach than SpaceX’s iterative approach. Each trial is created from scratch. Researchers develop a protocol custom-designed to study a specific drug. Then, infrastructure, researchers, and contractors are all onboarded for that specific trial.
Finally, that one-of-a-kind trial is executed, carrying its precious cargo: the pharmaceutical company’s treasured pharmaceutical asset, with billions of dollars of revenue on the line.
This again seems obviously crazy. There should be a standard product called a clinical trial, you should do it in standard ways over and over again.
Why no fixes?
Could one of the incumbents in this industry break through this morass? It’s unlikely because there are no players in the industry with the ability, resources, and incentive to drive improvement.
It starts with a lack of motivation. Pharmaceutical companies don’t face competitive pressure over trial costs.
… But let’s say a company were aware of these problems and genuinely wanted to do something about them. They still might lack the internal capacity to do so.
… The clinical trials industry has a culture problem. In short, they don’t sufficiently acknowledge that they are an industry in the first place.
… The result: leaders and policymakers don’t apply an engineering mindset to trials.
I don’t buy the ‘no motivation’ argument, as reduced costs drive more profits, and there are at least several big players who do enough iterating to justify this, and if they figured it out they could turn the trials into a new profit center.
This feels like a clear ‘if he wanted to’ situation. You can fix the cultural issue if you care enough. The real problem is indeed cultural, and it’s that they consider everything involved sacred, and everyone demands the everything bagels at all times, and those are sufficient legal and other barriers that no one dares try to fix things.
Why not let researchers choose their own accredited IRB, at the risk of taking some of the internal out of the review board? This is very much a ‘mostly disarm the IRBs’ proposal and I fully support it. That’s because I think IRBs are totally out of control and cause orders of magnitude more harm than they prevent, so letting people shop for the most efficient and accommodating one seems fine. The point would be that someone independent of you has to put themselves and their reputation on the line to say you’re doing something reasonable and not doing any blatant ethical violations, whereas right now no one can do much of anything.
A new large observational study claims that 45 minutes a day of walking gives you most of the all-cause mortality benefits of exercise, and running doesn’t do better than walking until you do rather a lot of it, whereas too much jogging can actively backfire and swimming and bicycling do little. Doing multiple types of activity was better.
Note that the X-axis on different graphs can be very different here.
It’s an observational and self-reported study, so you can’t assume causation and doubtless ‘willing and able’ correlations should be doing a lot of work. Presumably people who swim or bicycle must have some real problems? But there’s still a lot of explaining to do and in some places that only raises further questions.
The obvious response is to defy the data, or at least some of it, or at least attribute it to selection effects. It doesn’t make any sense for bicycling and swimming to have about zero effect and jogging to start actively backfiring this much, while walking cuts mortality by 20%.
I do buy that this is evidence for ‘if you do a lot of walking that largely counts being active and doing exercise.’
Being in generally poor health causes all sorts of other issues, part quite a lot:
Not a Good Jewish Girl: My fiancée is extremely mad that losing 20 pounds with a GLP-1 has basically cured her sleep and back issues and made her PMS symptoms much more tolerable, improving her overall mood on a daily basis and making her job less stressful.
These drugs should be fully subsidized.
The obvious way to do that is to buy one of the drugs and give it away, or sell it as a generic at marginal cost. It’s fine to pay the NPV of future royalties to get it, it’s a win-win, and you only have to buy one of them so you can play them off against each other.
Arthur B.: The US government could buy the patent for some oral GLP agonist like orforglipron for under $30B, give it away with Medicaid and save about $200B a year in Medicaid / Medicare future spending on obesity-linked diseases.
Or the market can solve this on its own, if this crosses the threshold and becomes worthwhile enough that consumers pay cash. Go go gadget market incentives.
David Wainer: Two years ago, a GLP-1 prescription could cost an uninsured patient more than $1,000 a month. Today, Novo Nordisk’s Wegovy pill starts at just $149 through cash-pay programs.
… Then on Thursday, shares of both companies fell sharply after Hims & Hers Health announced a $49-a-month compounded version of the Wegovy pill. They rebounded Friday after the Food and Drug Administration chief threatened action against the mass marketing of copycat drugs.
Enguerrand VII de Coucy: I find it difficult to understand how this could have happened, it’s almost like a force that we can’t see intervened in the GLP-1 market to bring prices down as unrestrained market competition increases… an invisible foot stomped down prices, perhaps? Translucent elbow?
I think it’s reasonable to be for or against assisted suicide to varying degrees. But if someone would be allowed to invoke MAID and kill themselves, then you have to also allow them (on their own dime as needed) to then do any other crazy thing that only might kill them. As Alex Tabarrok puts it: If you have the right to die, you should have the right to try, at least if you absolve the system of your future oversized directly related medical expenses.
Scaramouche: My doctor told me ethics approval keep letting his patients die so I asked how many patients he has and he said he just goes to the shelter and gets a new one afterwards so I said it sounds like he’s just feeding cancer patients to ethics and then his daughter started crying
