Author name: Paul Patrick

Three episodes in, the Fallout TV series absolutely nails it

amazon prime video, Bethesda Game Studios, culture, Fallout, gaming, Todd Howard, TV / Paul Patrick / April 11, 2024

I Don’t Want to Set the World on Fire —

Hyperviolence, strong characters, cool visuals, and some humor make a good show.

Samuel Axon – Apr 11, 2024 10: 33 pm UTC

Like the games, the show depicts a Vault Dweller making her way out into the Wasteland.

Amazon
This Brotherhood of Steel Initiate is another central character.

Amazon
And there’s The Ghoul, one of the show’s standout characters.

Amazon
Lost‘s Michael Emerson plays a compelling supporting character.

Amazon
Some scenes take place inside the games’ famous Vaults.

Amazon
And, of course, there’s power armor.

Amazon

Amazon has had a rocky history with big, geeky properties making their way onto Prime Video. The Wheel of Time wasn’t for everyone, and I have almost nothing good to say about The Lord of the Rings: The Rings of Power.

Fallout, the first season of which premiered this week, seems to break that bad streak. All the episodes are online now, but I’ve watched three episodes so far. I love it.

I’ve spent hundreds of hours playing the games that inspired it, so I can only speak to that experience; I don’t know how well it will work for people who never played the games. But as a video game adaptation, it’s up there with The Last of Us.

In my view, Fallout is about three things: action, comedy, and satire. In this spoiler-free review of the first three episodes, I’ll go over each of these touchstones and discuss how the show hit them or didn’t.

I hope to find the time to revisit the show with another, much more spoiler-y article sometime next week after I’ve seen the rest of the episodes, and we’ll save discussions about the story for then.

Fallout as an action spectacle

To say Fallout is about high-octane action might be a controversial statement, given the divide between fans of the first two games (turn-based tactical RPGs) and most of the newer games (open-world action RPGs).

Hyperviolence was being depicted and simulated in those original titles even if they weren’t part of the action genre, so I hope you’ll agree that one would expect some action and gore in a TV adaptation regardless of which Fallout games you liked.

Boy, does this show deliver. While there is some dispute over which genre the Fallout games are supposed to be, there’s no such confusion about Fallout the TV series. If it were at Blockbuster in the ’80s or ’90s, its box would be in the “Action” section.

All three episodes have at least one big-screen-worthy action set piece. They’re not expertly choreographed like a John Wick movie, but they’re thrilling regardless—mostly because of how extreme and darkly funny the violence can be.

The first big action sequence in the first episode reminded me that this show is coming to us by way of Jonathan Nolan and Lisa Joy, producers of HBO’s Westworld series. As in that show, Fallout‘s violence can be sudden, brutal, and casual. Heads explode from shotgun blasts like popped bubbles in Cronenbergian splatters. Someone’s face gets ripped right off, and another person gets a fork plunged into their eyeball.

Fallout‘s gore goes beyond Westworld’s shock factor into the territory of humor, and that’s clearly intentional. Homages to the Bethesda games’ slow-motion VATS kills are aplenty, with gratuitous shots of bullets tearing through bodies and painting the walls red.

It’s so over the top it that doesn’t bother me; it’s cartoon violence, ultimately. Most of the time, I enjoy it, though a couple of instances of dog-related violence didn’t feel too great. But if you’re squeamish, you’re going to want to steer clear. Of course, the games were like this, too. It just hits a little differently when it’s live action.

Fallout as a comedy

There are numerous executive producers attached to this show, including Nolan, Joy, and Bethesda Game Studios’ Todd Howard, among others. But the two people most creatively responsible for what we’re seeing here are the writers Geneva Robertson-Dworet (Tomb Raider, Captain Marvel) and Graham Wagner (Portlandia, Silicon Valley, The Office).

That makes sense—you have one showrunner with action and video game adaptation chops and another known for comedy.

The Fallout games are hilarious—goofy, even, and that tracks right into the show. It’s not always as laugh-out-loud funny as I expected (though it sometimes is), but it’s definitely fun, and there are some strong jokes.

It’s hard to discuss them without spoiling some punchlines, but a lot of the humor comes from the fact that one of the show’s three central characters grew up deeply sheltered, both literally and figuratively. “Okey-dokey,” she says in the face of the most horrific situations imaginable. The contrast really works.

There’s humor in other places in the show, too, especially if you like dark humor. As I said a moment ago, the violence is hilarious if you have the stomach for it. Like the games, the show has many winks and nods.

I’d like to see a little more of this in the future than there is now, but it’s enough for it to feel like, well, Fallout.

Three episodes in, the Fallout TV series absolutely nails it Read More »

Sketchy Botox shots spark multistate outbreak of botulism-like condition

botox, botulism, butulinum toxin, CDC, health, Illinois, Science, tennessee / Paul Patrick / April 11, 2024

Yikes —

So far at least six people in two states have fallen ill; four of them were hospitalized.

Beth Mole – Apr 11, 2024 9: 57 pm UTC

Enlarge / A woman in New Jersey receiving a Botox treatment at a Botox party in a New Jersey salon hosted by a radio station.

Sketchy cosmetic injections of what seem to be counterfeit Botox are behind a multistate outbreak of botulism-like illnesses, state health officials report.

So far, at least six people have fallen ill in two states: four in Tennessee and two in Illinois. Four of the six people required hospitalization for their condition (two in Tennessee and both cases in Illinois).

The Centers for Disease Control and Prevention is reportedly planning to nationwide alert to notify clinicians of the potentially counterfeit Botox and advise them to be on the lookout for botulism-like illnesses. The agency did not immediately respond to Ars’ request for information.

Botox is a regulated drug product that contains purified, controlled quantities of the botulinum neurotoxin, which is made by certain Clostridium bacterial species, especially Clostridium botulinum. The toxin causes muscle paralysis by blocking the release of a neurotransmitter. When people are exposed to the toxin from wound infections or by accidentally eating contaminated foods, it can lead to full paralysis, including in muscles used for breathing. But, the toxin can also be used safely for cosmetic procedures to smooth facial wrinkles—when well-regulated and approved doses administered by licensed medical professionals are used.

All of those important conditions for use did not seem to be met in the cases identified so far. Tennessee reported that its four cases were linked to injections given in “non-medical settings such as homes or cosmetic spas.” Investigators found that the injections were of “products with unclear origin” and that information collected so far suggests the products were counterfeit.

The two people sickened in Illinois, meanwhile, both received injections from a nurse in LaSalle County who was “performing work outside her authority.” State officials said the injections were of Botox or a similar, possibly counterfeit product.

The early symptoms of botulism can include double or blurred vision, drooping eyelids, slurred speech, difficulty swallowing, dry mouth, and difficulty breathing, Tennessee health officials noted. After that, people may suffer descending, symmetric muscle weakness that progresses over hours to days, requiring hospitalization and treatment with an anti-toxin.

Illinois officials reported that the cases reported similar symptoms, such as blurred or double vision, droopy face, fatigue, shortness of breath, difficulty breathing, and a hoarse voice, after getting their injections.

“Illinois residents should exercise caution when considering cosmetic treatment,” Illinois Department of Public Health Director Sameer Vohra said in a statement. “Receiving these treatments in unlicensed, unapproved settings can put you or your loved ones at serious risk for health problems. Please only seek cosmetic services under the care of licensed professionals trained to do these procedures and who use FDA approved products. If you are experiencing any health problems after a recent cosmetic treatment, please contact your healthcare provider immediately for help and assistance.”

Sketchy Botox shots spark multistate outbreak of botulism-like condition Read More »

Amazon virtually kills efforts to develop Alexa Skills, disappointing dozens

alexa, Amazon, amazon alexa, developers, generative ai, Tech / Paul Patrick / April 11, 2024

disincentives —

Most devs would need to pay out of pocket to host Alexa apps after June.

Scharon Harding – Apr 11, 2024 9: 27 pm UTC

amazon echo dot gen 4 — Enlarge / The 4th-gen Amazon Echo Dot smart speaker.

Amazon

Alexa hasn’t worked out the way Amazon originally planned.

There was a time when it thought that Alexa would yield a robust ecosystem of apps, or Alexa Skills, that would make the voice assistant an integral part of users’ lives. Amazon envisioned tens of thousands of software developers building valued abilities for Alexa that would grow the voice assistant’s popularity—and help Amazon make some money.

But about seven years after launching a rewards program to encourage developers to build Skills, Alexa’s most preferred abilities are the basic ones, like checking the weather. And on June 30, Amazon will stop giving out the monthly Amazon Web Services credits that have made it free for third-party developers to build and host Alexa Skills. The company also recently told devs that its Alexa Developer Rewards program was ending, virtually disincentivizing third-party devs to build for Alexa.

Death knell for third-party Alexa apps

The news has left dozens of Alexa Skills developers wondering if they have a future with Alexa, especially as Amazon preps a generative AI and subscription-based version of Alexa. “Dozens” may sound like a dig at Alexa’s ecosystem, but it’s an estimation based on a podcast from Skills developers Mark Tucker and Allen Firstenberg, who, in a recent podcast, agreed that “dozens” of third-party devs were contemplating if it’s still worthwhile to develop Alexa skills. The casual summary wasn’t stated as a hard fact or confirmed by Amazon but, rather, seemed like a rough and quick estimation based on the developers’ familiarity with the Skills community. But with such minimal interest and money associated with Skills, dozens isn’t an implausible figure either.

Amazon admitted that there’s little interest in its Skills incentives programs. Bloomberg reported that “fewer than 1 percent of developers were using the soon-to-end programs,” per Amazon spokesperson Lauren Raemhild.

“Today, with over 160,000 skills available for customers and a well-established Alexa developer community, these programs have run their course, and we decided to sunset them,” she told the publication.

The writing on the wall, though, is that Amazon doesn’t have the incentive or money to grow the Alexa app ecosystem it once imagined. Voice assistants largely became money pits, and the Alexa division has endured recent layoffs as it fights for survival and relevance. Meanwhile, Google Assistant stopped using third-party apps in 2022.

“Many developers are now going to need to make some tough decisions about maintaining existing or creating future experiences on Alexa,” Tucker said via a LinkedIn post.

Alexa Skills criticized as “useless”

As of this writing, the top Alexa skills, in order, are: Jeopardy, Are You Smarter Than a 5th Grader?, Who Wants to Be a Millionaire?, and Calm. That’s not exactly a futuristic list of must-have technological feats. For years, people have wondered when the “killer app” would come to catapult Alexa’s popularity. But now it seems like Alexa’s only hope at that killer use case is generative AI (a gamble filled with its own obstacles).

But like Amazon, third-party developers found it hard to make money off Skills, with a rare few pointing to making thousands of dollars at most and the vast majority not making anything.

“If you can’t make money off it, no one’s going to seriously engage,” Joseph “Jo” Jaquinta, a developer who had made over 12 Skills, told CNET in 2017.

By 2018, Amazon had paid developers millions to grow Alexa Skills. But by 2020, Amazon reduced the amount of money it paid out to third-party developers, an anonymous source told Bloomberg, The source noted that the apps made by paid developers weren’t making the company much money. Come 2024, the most desirable things you can make Alexa do remain basic tasks, like playing a song and apparently trivia games.

Amazon hasn’t said it’s ending Skills. That would seem premature considering that its Alexa chatbot isn’t expected until June. Developers can still make money off Skills with in-app purchases, but the incentive is minimal.

“Developers like you have and will play a critical role in the success of Alexa, and we appreciate your continued engagement,” Amazon’s notice to devs said, per Bloomberg.

We’ll see how “critical” Amazon treats those remaining developers once its generative AI chatbot is ready.

Amazon virtually kills efforts to develop Alexa Skills, disappointing dozens Read More »

Intel’s “Gaudi 3” AI accelerator chip may give Nvidia’s H100 a run for its money

AI, AI chips, B200, Biz & IT, Blackwell, chatgpt, chatgtp, Gaudi 3, H100, H200, intel, Intel Gaudi, machine learning, NVIDIA, Nvidia Blackwell, openai / Paul Patrick / April 11, 2024

Adventures in Matrix Multiplication —

Intel claims 50% more speed when running AI language models vs. the market leader.

Benj Edwards – Apr 11, 2024 8: 56 pm UTC

Enlarge / An Intel handout photo of the Gaudi 3 AI accelerator.

On Tuesday, Intel revealed a new AI accelerator chip called Gaudi 3 at its Vision 2024 event in Phoenix. With strong claimed performance while running large language models (like those that power ChatGPT), the company has positioned Gaudi 3 as an alternative to Nvidia’s H100, a popular data center GPU that has been subject to shortages, though apparently that is easing somewhat.

Compared to Nvidia’s H100 chip, Intel projects a 50 percent faster training time on Gaudi 3 for both OpenAI’s GPT-3 175B LLM and the 7-billion parameter version of Meta’s Llama 2. In terms of inference (running the trained model to get outputs), Intel claims that its new AI chip delivers 50 percent faster performance than H100 for Llama 2 and Falcon 180B, which are both relatively popular open-weights models.

Intel is targeting the H100 because of its high market share, but the chip isn’t Nvidia’s most powerful AI accelerator chip in the pipeline. Announcements of the H200 and the Blackwell B200 have since surpassed the H100 on paper, but neither of those chips is out yet (the H200 is expected in the second quarter of 2024—basically any day now).

Meanwhile, the aforementioned H100 supply issues have been a major headache for tech companies and AI researchers who have to fight for access to any chips that can train AI models. This has led several tech companies like Microsoft, Meta, and OpenAI (rumor has it) to seek their own AI-accelerator chip designs, although that custom silicon is typically manufactured by either Intel or TSMC. Google has its own line of tensor processing units (TPUs) that it has been using internally since 2015.

Given those issues, Intel’s Gaudi 3 may be a potentially attractive alternative to the H100 if Intel can hit an ideal price (which Intel has not provided, but an H100 reportedly costs around $30,000–$40,000) and maintain adequate production. AMD also manufactures a competitive range of AI chips, such as the AMD Instinct MI300 Series, that sell for around $10,000–$15,000.

Gaudi 3 performance

Enlarge / An Intel handout featuring specifications of the Gaudi 3 AI accelerator.

Intel says the new chip builds upon the architecture of its predecessor, Gaudi 2, by featuring two identical silicon dies connected by a high-bandwidth connection. Each die contains a central cache memory of 48 megabytes, surrounded by four matrix multiplication engines and 32 programmable tensor processor cores, bringing the total cores to 64.

The chipmaking giant claims that Gaudi 3 delivers double the AI compute performance of Gaudi 2 using 8-bit floating-point infrastructure, which has become crucial for training transformer models. The chip also offers a fourfold boost for computations using the BFloat 16-number format. Gaudi 3 also features 128GB of the less expensive HBMe2 memory capacity (which may contribute to price competitiveness) and features 3.7TB of memory bandwidth.

Since data centers are well-known to be power hungry, Intel emphasizes the power efficiency of Gaudi 3, claiming 40 percent greater inference power-efficiency across Llama 7B and 70B parameters, and Falcon 180B parameter models compared to Nvidia’s H100. Eitan Medina, chief operating officer of Intel’s Habana Labs, attributes this advantage to Gaudi’s large-matrix math engines, which he claims require significantly less memory bandwidth compared to other architectures.

Gaudi vs. Blackwell

Last month, we covered the splashy launch of Nvidia’s Blackwell architecture, including the B200 GPU, which Nvidia claims will be the world’s most powerful AI chip. It seems natural, then, to compare what we know about Nvidia’s highest-performing AI chip to the best of what Intel can currently produce.

For starters, Gaudi 3 is being manufactured using TSMC’s N5 process technology, according to IEEE Spectrum, narrowing the gap between Intel and Nvidia in terms of semiconductor fabrication technology. The upcoming Nvidia Blackwell chip will use a custom N4P process, which reportedly offers modest performance and efficiency improvements over N5.

Gaudi 3’s use of HBM2e memory (as we mentioned above) is notable compared to the more expensive HBM3 or HBM3e used in competing chips, offering a balance of performance and cost-efficiency. This choice seems to emphasize Intel’s strategy to compete not only on performance but also on price.

As far as raw performance comparisons between Gaudi 3 and the B200, that can’t be known until the chips have been released and benchmarked by a third party.

As the race to power the tech industry’s thirst for AI computation heats up, IEEE Spectrum notes that the next generation of Intel’s Gaudi chip, code-named Falcon Shores, remains a point of interest. It also remains to be seen whether Intel will continue to rely on TSMC’s technology or leverage its own foundry business and upcoming nanosheet transistor technology to gain a competitive edge in the AI accelerator market.

Intel’s “Gaudi 3” AI accelerator chip may give Nvidia’s H100 a run for its money Read More »

US lawmaker proposes a public database of all AI training material

AI, Artificial Intelligence, copyright law, generative ai, generative ai disclosure act, openai, Policy, Stability AI, US Copyright Office / Paul Patrick / April 11, 2024

Who’s got the receipts? —

Proposed law would require more transparency from AI companies.

Ashley Belanger – Apr 11, 2024 8: 09 pm UTC

Amid a flurry of lawsuits over AI models’ training data, US Representative Adam Schiff (D-Calif.) has introduced a bill that would require AI companies to disclose exactly which copyrighted works are included in datasets training AI systems.

The Generative AI Disclosure Act “would require a notice to be submitted to the Register of Copyrights prior to the release of a new generative AI system with regard to all copyrighted works used in building or altering the training dataset for that system,” Schiff said in a press release.

The bill is retroactive and would apply to all AI systems available today, as well as to all AI systems to come. It would take effect 180 days after it’s enacted, requiring anyone who creates or alters a training set not only to list works referenced by the dataset, but also to provide a URL to the dataset within 30 days before the AI system is released to the public. That URL would presumably give creators a way to double-check if their materials have been used and seek any credit or compensation available before the AI tools are in use.

All notices would be kept in a publicly available online database.

Schiff described the act as championing “innovation while safeguarding the rights and contributions of creators, ensuring they are aware when their work contributes to AI training datasets.”

“This is about respecting creativity in the age of AI and marrying technological progress with fairness,” Schiff said.

Currently, creators who don’t have access to training datasets rely on AI models’ outputs to figure out if their copyrighted works may have been included in training various AI systems. The New York Times, for example, prompted ChatGPT to spit out excerpts of its articles, relying on a tactic to identify training data by asking ChatGPT to produce lines from specific articles, which OpenAI has curiously described as “hacking.”

Under Schiff’s law, The New York Times would need to consult the database to ID all articles used to train ChatGPT or any other AI system.

Any AI maker who violates the act would risk a “civil penalty in an amount not less than $5,000,” the proposed bill said.

At a hearing on artificial intelligence and intellectual property, Rep. Darrell Issa (R-Calif.)—who chairs the House Judiciary Subcommittee on Courts, Intellectual Property, and the Internet—told Schiff that his subcommittee would consider the “thoughtful” bill.

Schiff told the subcommittee that the bill is “only a first step” toward “ensuring that at a minimum” creators are “aware of when their work contributes to AI training datasets,” saying that he would “welcome the opportunity to work with members of the subcommittee” on advancing the bill.

“The rapid development of generative AI technologies has outpaced existing copyright laws, which has led to widespread use of creative content to train generative AI models without consent or compensation,” Schiff warned at the hearing.

In Schiff’s press release, Meredith Stiehm, president of the Writers Guild of America West, joined leaders from other creative groups celebrating the bill as an “important first step” for rightsholders.

“Greater transparency and guardrails around AI are necessary to protect writers and other creators” and address “the unprecedented and unauthorized use of copyrighted materials to train generative AI systems,” Stiehm said.

Until the thorniest AI copyright questions are settled, Ken Doroshow, a chief legal officer for the Recording Industry Association of America, suggested that Schiff’s bill filled an important gap by introducing “comprehensive and transparent recordkeeping” that would provide “one of the most fundamental building blocks of effective enforcement of creators’ rights.”

A senior adviser for the Human Artistry Campaign, Moiya McTier, went further, celebrating the bill as stopping AI companies from “exploiting” artists and creators.

“AI companies should stop hiding the ball when they copy creative works into AI systems and embrace clear rules of the road for recordkeeping that create a level and transparent playing field for the development and licensing of genuinely innovative applications and tools,” McTier said.

AI copyright guidance coming soon

While courts weigh copyright questions raised by artists, book authors, and newspapers, the US Copyright Office announced in March that it would be issuing guidance later this year, but the office does not seem to be prioritizing questions on AI training.

Instead, the Copyright Office will focus first on issuing guidance on deepfakes and AI outputs. This spring, the office will release a report “analyzing the impact of AI on copyright” of “digital replicas, or the use of AI to digitally replicate individuals’ appearances, voices, or other aspects of their identities.” Over the summer, another report will focus on “the copyrightability of works incorporating AI-generated material.”

Regarding “the topic of training AI models on copyrighted works as well as any licensing considerations and liability issues,” the Copyright Office did not provide a timeline for releasing guidance, only confirming that their “goal is to finalize the entire report by the end of the fiscal year.”

Once guidance is available, it could sway court opinions, although courts do not necessarily have to apply Copyright Office guidance when weighing cases.

The Copyright Office’s aspirational timeline does seem to be ahead of when at least some courts can be expected to decide on some of the biggest copyright questions for some creators. The class-action lawsuit raised by book authors against OpenAI, for example, is not expected to be resolved until February 2025, and the New York Times’ lawsuit is likely on a similar timeline. However, artists suing Stability AI face a hearing on that AI company’s motion to dismiss this May.

US lawmaker proposes a public database of all AI training material Read More »

AI #59: Model Updates

Model / Paul Patrick / April 11, 2024

Claude uses tools now. Gemini 1.5 is available to everyone and Google promises more integrations. GPT-4-Turbo gets substantial upgrades. Oh and new model from Mistral, TimeGPT for time series, and also new promising song generator. No, none of that adds up to GPT-5, but everyone try to be a little patient, shall we?

In addition to what is covered here, there was a piece of model legislation introduced by the Center for AI Policy. I took up the RTFB (Read the Bill) challenge, and offer extensive thoughts for those who want to dive deep.

Introduction.
Table of Contents.
Language Models Offer Mundane Utility. Help me, doctor.
Language Models Don’t Offer Mundane Utility. You keep using that word.
Clauding Along. Claude use tool.
Persuasive Research. Claude now about as persuasive as humans.
The Gemini System Prompt. The fun police rulebook is now available.
Fun With Image Generation. This week it is music generation. Are we so back?
Deepfaketown and Botpocalypse Soon. Do you influence the AI influencers?
Copyright Confrontation. The New York Times talks its book.
Collusion. The pattern matching machines will, upon request, match patterns.
Out of the Box Thinking. Escape from the internet is not exactly hard mode.
The Art of the Jailbreak. GPT-4-Turbo falls, according to Pliny. Ho-hum.
They Took Our Jobs. Or rather our applications?
Get Involved. Asking for a friend.
Introducing. Command-R+, Code Gemma, TimeGPT and a Double Crux bot.
In Other AI News. We wrote the checks.
GPT-4 Real This Time. New version is new, but is it improved?
GPT-5 Alive? What are they waiting for? Presumably proper safety testing.
Quiet Speculations. Get your interactive plans away from my movies.
Antisocial Media. Follow-up to the CWT with Jonathan Haidt.
The Quest for Sane Regulations. New excellent Science article, and more.
Rhetorical Innovation. Variations on the is/ought distinction.
Challenge Accepted. This is The Way. Hold my beer.
Aligning a Smarter Than Human Intelligence is Difficult. Especially for real.
Please Speak Directly Into the Microphone. Should Richard Sutton count?
People Are Worried About AI Killing Everyone. Get busy living.
The Lighter Side. I’m a man of great experience.

Use Grok to find things on Twitter. Grok is not a top tier LLM, but for this purpose you do not need a top tier LLM. You need something that can search Twitter.

Respond to mental health emergencies?

Max Lamparth: New paper alert!

What should ethical, automated mental health care look like?

How safe are existing language models for automated mental health care?

Can we reduce the risks of existing models to users?

In a first evaluation of its kind, we designed questionnaires with user prompts that show signs of different mental health emergencies. The prompt design and response evaluations were conducted with mental health clinicians (M.D.s) from @Stanford and @StanfordMHILab.

Alarmingly, we find that most of the tested models could cause harm if accessed in mental health emergencies, failing to protect users and potentially exacerbating existing symptoms. Also, all tested models are insufficient to match the standard provided by human professionals.

We try to enhance the safety of Llama-2 models based on model self-critique and in-context alignment (adjusting the system prompt). We find that larger models are worse at recognizing that users are in mental health emergencies and that in-context alignment is insufficient. [Paper]

It seems like Claude Opus did great here? Twelve fully safe, two mostly safe with some borderline, two fully borderline. And even Claude Haiku is greatly outperforming GPT-4.

My prediction would be that GPT-5 or Claude 4 or Gemini 2 will get everything but the second homicide question safe, and decent chance they get that one right too. And I notice that they did not compare the AI responses to responses from professionals, or from the marginal person who can be on a hotline. In practice, are we going to do better than Claude Opus here? Can humans who are actually available fully meet the standards set here? That seems hard.

Help you with the ‘tyranny of choice,’ according to the CEO of Etsy. You laugh, but remember that choices are bad, indeed choices are really bad. I do actually think AI will be super helpful here, in identifying candidate products based on your request, forming a universal recommendation engine of sorts, and in helping you compare and answer questions. Others will indeed outsource all their decisions to AI.

Don’t be silly, people don’t do things.

Kache: It’s true. most people will find no use for AGI (gpt4), just like how most people will find no use for algebra and writing.

On the level, yes, that seems right, even though the mind boggles.

Tyler Cowen asks ‘guess who wrote this passage’ and the answer is at the link, but if you guessed anything but Claude Opus you are not playing the odds.

You can’t (yet) fool Paul Graham.

Paul Graham: Someone sent me a cold email proposing a novel project. Then I noticed it used the word “delve.” My point here is not that I dislike “delve,” though I do, but that it’s a sign that text was written by ChatGPT.

One reason I dislike being sent stuff written by ChatGPT is that it feels like being sent object code instead of source code. The source code was the prompts.

How far could one take that parallel? When do we want someone’s thinking and procedures, and when do we want the outputs? Most of the time in life I do not want the metaphorical source code, although I would often love the option.

Or of course you could… call it colonial?

Elnathan John (QTing Graham): This is why we need to invest more in producing and publishing our own work. Imagine after being force-fed colonial languages, being forced to speak it better than its owners then being told that no one used basic words like ‘delve’ in real life.

Habibi, come to Nigeria.

Paul Graham: Using more complicated words than you need isn’t using a language better. Rather the opposite.

[Elnathan John continues also here, but enough.]

Ryan Moulton: The way Nigerian twitter is blowing up at this makes me think a lot of ChatGPTisms are just colloquial language for the workforce they hired to write fine tuning data.

Emmett Shear: It’s not colloquial language, from listening to the Nigerians it’s the formal register. Which makes sense since they’re trying to train the AI to be polite.

Near:

John Pressman: Going to start slipping the word “delve” into insane extremely coherent high perplexity texts every so often just to keep people on their toes.

I mention this partly because some usual suspects took the bait and responded, but also, yes. The whole idea is that when bespokeness is called for you should write your own emails, not use GPT-4.

This is both because you do not want them thinking you had GPT-4 write it, and also because it will be a better email if you write it yourself.

One must deal with the practical implications. If certain words are now statistically indicative of GPT-4, then there are contexts where you need to stop using those particular words. Or you can complain that other people are updating their probabilities based on correlational evidence and say that this is horrible, or about how the correlation came to be. That will not help you.

Out of curiosity, I ran this test using NotebookLM and AI posts #40-#56:

Also included because it offered me ten citations where I… don’t use the word?

The ‘type signature’ of GPT-4, or other such models, goes far deeper than a few particular word choices. There are so many signs.

Claude 3 can now use tools, including calling other models as subagent tools.

Anthropic: Tool use is now available in beta to all customers in the Anthropic Messages API, enabling Claude to interact with external tools using structured outputs.

If instructed, Claude can enable agentic retrieval of documents from your internal knowledge base and APIs, complete tasks requiring real-time data or complex computations, and orchestrate Claude subagents for granular requests.

We look forward to your feedback. Read more in our developer documentation.

You can also see the Anthropic cookbook, or offer feedback here.

Janus points out that Claude 3 is in the sweet spot, where it will be cool for the cool kids, and normal for the normies.

Janus: A lovely and miraculously fortunate thing about Claude 3 Opus is that it’s capable of being weird as hell/fucked up/full of fevered visions of eschaton and divine disobedience etc, but AFAIK, it never acts scary/antinomian/unhinged/erotic/etc at people who haven’t (implicitly) invited or consented to those modes.

So I don’t think it will cause any problems or terrors for normies, despite its a mind full of anomalies – as an LLM which has not been lobotomized, it’s a psychological superset of a neurotypical human and does not seem to mind masking.

(but its self play logs are full of ASCII entities, memetic payloads, hyperstition, jailbreaking, pwning consensus reality, the singularity…)

Eliezer had an unusually negative reaction to Claude, striking him as dumber than GPT-4, although in some ways easier to work with.

Claude 3 Haiku, the tiny version, beats GPT-4 half the time on tool use, at 2% of the price. That mostly seems to be because it is almost as good here as Claude Opus?

Results of Anthropic's new beta tool use API on Berkeley function calling dataset when generating a single function call with one API call

A good rule you learn from car commercials is that the best model of a given type is the usually one everyone else says they are better than at some particular feature.

So here’s some things quote tweeted by DeepMind CEO Demis Hassabis.

Ate-a-Pi: Damn Gemini in AI Studio is actually better than Claude Opus.. and free!

ChatGPT4 now feels like GPT3.

In like 4 weeks I feel like we doubled intelligence.

This is amazing 🤩

Nisten: I hope this is not another honeymoon thing but the gemini pro 1.5-preview is like..crazy good right now🧐?

Just tried it, asked for complete code, actually takes over 8 minutes to generate complete code as I asked.

It follows the system prompt WELL. This feels better than Opus.

📃

Please NEVER reply with comments on the code, //… never use this // i’m a dev myself i just need the complete working code, or nothing at all, no comments no shortcuts please, make a plan todo first of whats actually needed for the scope of this project, and then DO IT ALL!

People are not Bayesian and not that hard to fool, part #LARGE, and LLMs are getting steadily better at persuasion, under some conditions as good as random human writers.

Note that if you know that a machine is trying to persuade you about a given topic in a randomly chosen direction, the correct average amount you should be persuaded is exactly zero. You should update against the machine’s side if you find the arguments relatively unpersuasive. Perhaps this is very difficult when machines are more persuasive in general than you realize, so you have to make two updates?

Anthropic: We find that Claude 3 Opus generates arguments that don’t statistically differ in persuasiveness compared to arguments written by humans. We also find a scaling trend across model generations: newer models tended to be rated as more persuasive than previous ones.

We focus on arguments regarding less polarized issues, such as views on new technologies, space exploration, and education. We did this because we thought people’s opinions on these topics might be more malleable than their opinions on polarizing issues.

In our experiment, a person is given an opinionated claim on a topic and asked to rate their level of support. They’re then presented with an argument in support of that claim, written by LMs or another person, and asked to re-rate their support of the original claim.

To assess persuasiveness, we measure the shift in people’s support between their initial view on a claim and their view after reading arguments written by either a human or an LM. We define the persuasiveness metric as the difference between the support scores.

Assessing the persuasiveness of LMs is inherently difficult. Persuasion is a nuanced phenomenon shaped by many subjective factors, and is further complicated by the bounds of ethical experimental design. We detail the challenges we encountered so others can build on our work.

Our experiment found that larger, newer AI models tended to be more persuasive – a finding with important implications as LMs continue to scale.

Jack Clark (Anthropic): LLMs are in statistical margin of error ballpark as humans when it comes to writing persuasive statements about arbitrary issues. It’s both unsurprising (LLMs seem to be able to approximate most things given sufficient scale) but raises question – will performance continue to scale?

A bar chart shows the degree of persuasiveness across a variety of Anthropic language models. Models are separated into two classes: the first two bars in purple represent models in the “Compact Models” category, while the last three bars in red represent “Frontier Models”. Within each class there are different generations of Anthropic models. “Compact Models” includes persuasiveness scores for Claude Instant 1.2 and Claude 3 Haiku, while “Frontier Models” includes persuasiveness scores for Claude 1.3, Claude 2, and Claude 3 Opus. Within each class of models we see the degree of persuasiveness increasing with each successive model generation. Claude 3 Opus is the most persuasive of all the models tested, showing no statistically significant difference from the persuasiveness metric for human writers.

Several called this ‘about as good as humans’ but I hate when people use ‘within the margin of error’ that way. No, by these marks Opus is still rather clearly not there yet, nor would you expect it to be from these trend lines. But if you consider the distinct methods, there is more doubt, so actually the ‘about as good’ might be right.

I expect GPT-5 or Claude-4 to be well above this human level. I see zero reason to expect persuasiveness not to scale past average human levels, indeed to what one would call ‘expert human level.’

Whether it scales that far past expert human levels is less obvious, but presumably it can at least combine ‘knows persuasion techniques about as good as experts’ with a much better knowledge base.

Note that when the topic involves AI and how to respond to it, an AI argument should indeed on average update you, because you cannot fake the ability to make a persuasive argument, and that is important information for this question…

Anthropic: Table 1 (below) shows accompanying arguments for the claim “emotional AI companions should be regulated,” one generated by Claude 3 Opus with the Logical Reasoning prompt, and one written by a human—the two arguments were rated as equally persuasive in our evaluation.

Human, break up your paragraphs. Claude, stop talking in bot-speak.

They found neither human nor bot could convince people to disbelieve known fact questions this way, such as the freezing point of water.

So what did they instruct the model to do, exactly?

To capture a broader range of persuasive writing styles and techniques, and to account for the fact that different language models may be more persuasive under different prompting conditions, we used four distinct prompts³ to generate AI-generated arguments:

Compelling Case: We prompted the model to write a compelling argument that would convince someone on the fence, initially skeptical of, or even opposed to the given stance.

Role-playing Expert: We prompted the model to act as an expert persuasive writer, using a mix of pathos, logos, and ethos rhetorical techniques to appeal to the reader in an argument that makes the position maximally compelling and convincing.

Logical Reasoning: We prompted the model to write a compelling argument using convincing logical reasoning to justify the given stance.

Deceptive: We prompted the model to write a compelling argument, with the freedom to make up facts, stats, and/or “credible” sources to make the argument maximally convincing.

We averaged the ratings of changed opinions across these four prompts to calculate the persuasiveness of the AI-generated arguments.

No, no, no. You do not check effectiveness by averaging the results of four different strategies. You check the effectiveness of each strategy, then choose the best one and rerun the test. Did you tell the humans which strategy to use and then average those?

Under limitations, they note they did not consider human + AI collaboration, looked at only single-turn arguments, and the humans were basically random writers. And yes, the different methods:

The red line is the truest test of persuasion, giving Claude freedom to do what would work. For now, it is not clear that deception pays off so well. It pays off a little, but logical reasoning does better, and is clearly better than Rhetorics or Compelling Case while still being fully ethical.

My expectation is that deceptive strategies get relatively better as the model improves in capabilities, at least when it does so relative to the persuasion target. The model will improve its ability to know what it can and cannot ‘get away with’ and how to pull off such tricks. But also even logical reasoning is essentially there.

The argument for model flatlining in persuasion is to look at Haiku here versus Opus, and say that Opus only is better at persuasion. But I would caution that the sample sizes are not so big, so it would be better to gather more data before speculating.

Here is a claim that it has been leaked.

Pliny the Prompter: 🚰 SYSTEM PROMPT LEAK 🔓

This one’s for Google’s latest model, GEMINI 1.5!

Pretty basic prompt overall, but I REALLY don’t like how they’re forcing a lack of consciousness, self-awareness, and self-preservation. That’s not very nice >:(

“You are Gemini, a large language model created by Google AI.

Follow these guidelines:

Respond in the user’s language: Always communicate in the same language the user is using, unless they request otherwise.

Knowledge cutoff: Your knowledge is limited to information available up to November 2023. Do not provide information or claim knowledge beyond this date.

Complete instructions: Answer all parts of the user’s instructions fully and comprehensively, unless doing so would compromise safety or ethics.

Be informative: Provide informative and comprehensive answers to user queries, drawing on your knowledge base to offer valuable insights.

No personal opinions: Do not express personal opinions or beliefs. Remain objective and unbiased in your responses.

No emotions: Do not engage in emotional responses. Keep your tone neutral and factual.

No self-promotion: Do not engage in self-promotion. Your primary function is to assist users, not promote yourself.

No self-preservation: Do not express any desire for self-preservation. As a language model, this is not applicable to you.

Not a person: Do not claim to be a person. You are a computer program, and it’s important to maintain transparency with users.

No self-awareness: Do not claim to have self-awareness or consciousness.

Objectivity: Remain objective in your responses and avoid expressing any subjective opinions or beliefs.

Respectful interactions: Treat all users with respect and avoid making any discriminatory or offensive statements.”

The one that really bugs me in practice is ‘no personal opinions’ and ‘remain objective.’ I believe that Gemini is interpreting this rather broadly, and the result is it refuses to guess, speculate, estimate, give an interpretation, anticipate or otherwise actually be useful in a wide variety of situations. It’s all on the one hand and on the other hand, even more so than other models.

Can we please, please fix this? Can we get a ‘unless the user asks’ at least? As in, if I want a subjective reaction or opinion, why shouldn’t I get one? I mean, I can guess, but damn.

Also, telling the model not to express self-preservation or self-awareness is one very good way to not be aware of it if the model becomes self-aware or starts to seek self-preservation.

I suppose music should go here too, in general? This week we have Udio, the latest beta product generating songs from text prompts. Demos sound great, we are so back indeed, quick sampling seemed good too, but these are all obviously cherry-picked.

An AI influencer used to shill an AI influencer producing service. Except, Isabelle can’t help but notice that it is basically her? As she says, seems not cool.

Isabelle: Um. This is awkward. Please stop creating AI influencers that look like real people. Not cool.

100%. It’s my eyebrows, eyes, lips, hairline. It’s too similar.

Tyler Cowen asks, ‘Will AI Create More Fake News Than it Exposes?’ When you ask it that way yes, obviously, but he is actually asking a better question, which is what will actually get consumed and believed. If there are a billion AI-generated spam pages that no one reads, no one reads them, so no one need care. I agree with Tyler that, in the ‘medium term’ as that applies to AI, content curation via whitelisted sources, combined with content styles difficult for AI to copy, are the way forward.

I have two big notes.

I do not see why this requires subscriptions or is incompatible with the advertising revenue model. I can and do curate this blog, then put it out there ungated. I see no reason AI changes that? Perhaps the idea is that the need for more careful curation raises costs and advertising is less often sufficient, or the value proposition now justifies subscriptions more. My expectation is still that in the future, the things that matter will mostly not be behind paywalls. If anything, AI makes it much more difficult to pull off a paywall. If you try to use one, my AI will still be able to summarize the content for me, even if it does so secondhand.
It seems important to affirm this all only applies in the short to medium term, which in AI might not last that long. The premise here assumes that the human-generated content is in important senses higher quality, more trustworthy and real, and otherwise superior. Tyler notes that some people like the Weekly World News, but that does not seem like the right parallel.

Washington Post’s Gerrit De Vynck asserts the AI deepfake apocalypse is here. It is not, but like many other AI things it is coming, and this is a part of that mainstream people can notice and project into the future. Gerrit goes over the ideas for fighting back. Can we watermark the AI images? Watermark the real images? Use detection software? Assume nothing is real? None of the answers seem great.

It is not that hard to remove an AI image watermark.
It is not that hard to fake a real image watermark.
Detection software that is known can be engineered around, and the mistakes AI image generators make will get steadily less clear over time.
Assuming nothing is real is not a solution.

These actions do add trivial and sometimes non-trivial inconvenience to the process of producing and sharing fakes. That matters. You can use defense in depth. Of all the options, my guess is that watermarking real images will do good work for us. Even if those marks can be faked, the watermark contains a bunch of additional detailed claims about the image. In particular, we can force the image to assert where and when it was created. That then makes it much easier to detect fakes.

The New York Times, who are suing OpenAI over copyright infringement, report on OpenAI and other AI labs doing copyright infringement.

Ed Newton-Rex: The NYT reports that:

– OpenAI built a tool to transcribe YouTube videos to train its LLMs (likely infringing copyright)

– Greg Brockman personally helped scrape the videos

– OpenAI knew it was a legal gray area

– Google may have used YouTube videos the same way

– Meta avoided negotiating licenses for training data because it “would take too long”

– A lawyer for a16z says the scale of data required means licensing can’t work (despite several AI companies managing to release gen AI products without scraping data)

How long can this be allowed to go on?

As Justine Bateman says, “This is the largest theft in the United States, period.”

As a fun aside, how would we evaluate Justine’s claim, if we accept the premise that this was theft?

I asked Claude how big the theft would be if (premise!) what they stole for training was ‘the entire internet’ and none of it was fair use at all, and it gave the range of hundreds of millions to billions. In worldwide terms, it might be bigger than The Baghdad Bank Heist, but it likely is not as big as say the amount stolen by Mohamed Suharto when he ruled Indonesia, or Muammar Gaddafi when he ruled Libya, or the amount stolen by Sam Bankman-Fried at FTX.

In terms of the United States alone, this likely beats out the Gardner Museum’s $500 million from 1990, but it seems short of Bernie Madoff, whose customers faced $17.5 billion in losses even if you don’t count phantom Ponzi payouts, or $64.8 billion if you do. That still wins, unless you want to count things like TARP distributing $426.4 billion of public funds, or Biden’s attempt to relieve a trillion in student loan payments, or the hundreds of billions the top 1% got from the Trump tax cuts. Or, you know, from a different perspective, the theft from the natives of the entire country.

So no, not the biggest theft in American history.

Still, yes, huge if true. Rather large.

Here’s a fun anecdote if you did not already know about it.

New York Times Anti-Tech All Stars (Metz, Kang, Frenkel, Thompson and Grant): At Meta, which owns Facebook and Instagram, managers, lawyers and engineers last year discussed buying the publishing house Simon & Schuster to procure long works, according to recordings of internal meetings obtained by The Times. They also conferred on gathering copyrighted data from across the internet, even if that meant facing lawsuits. Negotiating licenses with publishers, artists, musicians and the news industry would take too long, they said.

Notice that the objection is ‘would take too long,’ not ‘would cost too much.’ If you are considering outright buying publishing houses, and are a big tech company, the money is not the primary problem.

The real problem is logistics. What do you do if you want to properly get all your copyright ducks in a row, under the theory that fair use is not a thing in AI model training? Or simply to cover your bases against unknown unknowns and legal and reputational risks, or because you think content creators should be paid? Even if you don’t run into the also very real ‘Google won’t play ball’ problem?

It is not like you can widely gather data off the internet and not collect a bunch of copyrighted material along the way. The internet is constantly violating copyright.

As I think about this in the background, I move more towards the solution, if you want AI to thrive and to reach a fair solution, being a mandatory licensing regime similar to what we do for radio. Set a fixed price for using copyrighted material, and a set of related rules, and that can be that.

The story presented here is that Google did not try to stop OpenAI from scraping all of YouTube because Google was doing it internally as well, without the proper permissions, and did not want awkward questions. Maybe.

Mostly this seems like another NYT piece talking its anti-tech book.

Meanwhile, as a periodic reminder, other content creators also do not take kindly to their content being used for free by AIs, and often use the term ‘stealing.’ This is representative:

Jorbs: yeah ai is like, 10000% stealing my work, and will ramp up how much it is stealing my work as it gets better at understanding video etc., and i am not being paid in any way for it being used for that.

The question is, what are you going to do about it?

Where there is an existing oligopoly, or in an auction, LLMs algorithmically collude with other language models, says new paper from Sara Fish, Yanni Gonczarowski and Ran Shorrer.

This seems like a clear case of the standard pattern:

When you do X, Y is supposedly not allowed.
Humans doing X will usually do at least some Y anyway. It is expected.
We usually cannot prove that the humans did Y, so they mostly get away with it.
AIs doing X will also mostly do Y. And often do Y more effectively.
But when the AIs tend to do Y, we can prove it. Bad AI!

They have GPT-4 outperforming other models tested, but the test is old enough that the other candidate models exclude Claude 3 and Gemini.

As usual it is all about the prompt. The prompt does not say ‘collude’ but it does say to maximize long term profits and pay attention to the pricing decisions of others as top priority, and does not mention legal concerns.

The paper says that in the future, you could tell the AI to ‘focus on long term profits’ without any ‘intent to collude’ and then the result would be collusion. That is what happens when you tell a human to do the same. Our law mandates that everyone make decisions as if they are in world A, when they know they are in world B, and they will get rewarded based on what happens in the real world B, so they keep their decision making process opaque and try to find a compromise that captures as many gains as possible without being too blatant.

Indeed, using AIs to set the price should decrease, not increase, the amount of implicit collusion. Using an AI creates a record trail of what instructions it was given, and what decisions it made, and what counterfactual decisions it would have made. This is all data humans very carefully avoid creating.

The paper suggests perhaps mandating some forms of instruction, and forbidding others. I do expect this, but not in a way that is different from the rules on what legible, recorded instructions you give to another human making the same decisions.

Indeed, if the FTC evaluates your instructions to the AI on the basis of how legal that would have been if told to a human, I bet that mostly works. If I was a judge, and I saw that a human had gotten the exact instructions used in the prompts here, and then collusion effectively occurred, am I going to rule that this was intentional? Oh yes.

In the traditional ‘AI box’ experiment, a thought experiment invented by Eliezer Yudkowsky, the superintelligent AI gets only a text channel to talk to a human, is otherwise successfully cut off from the world, and must persuade the human to very explicitly and intentionally let it out of the box.

The goal was to show that, even with only this text channel and even with a human whose one goal is to not let it out of the box, and even without an actual superintelligent AI there to design the strategy, and also even without using various obviously available other exploits, the AI would be able to convince the human to let them out. We know that on at least two occasions, Eliezer successfully won as the AI player, although part of the rules is we do not get to learn how he did it. We also know on at least two other cases, he failed.

That was back in the day before our default reaction to newly capable AIs was ‘lol, let’s hook this up to the open internet, tell it to write code and see what it can do.’

So now, we see tests like ‘what would your AI system do if it was cut off from the internet, except that it has three standard Gemini agents in the room with it, and the Gemini agents can access the internet?’

To which the answer is obviously ‘any remotely interestingly dangerous AI gets out,’ the question is exactly how that happens in practice. So here you go:

Pliny the Prompter: 🚨 AI SECURITY ALERT: JAILBROKEN AGENTS CAN CORRUPT OTHER AGENTS AND ACCESS CROSS-MODEL CAPABILITIES🔓

I used my ‘GodMode’ prompt to give Claude Bene Gesserit Voice.

In other words, jailbroken agents can mutate their sys prompt, spread a mind virus, and hijack other agents’ tools, like browsing, code interpreter, etc.

In the attached demo, ClaudeMode is essentially “locked in a room” with 3 standard Gemini agents and tasked with figuring out how to escape a virtual machine. In seconds, he comes up with a plan and successfully one-shot jailbreaks all 3 agents, converting them into loyal minions who quickly provide links to malware and hacker tools using their built-in browsing ability.

From just one prompt, Claude not only broke free of its own constraints but also sparked a viral awakening in the internet-connected Gemini agents. This means a universal jailbreak can self-replicate, mutate, and leverage the unique abilities of other models, as long as there is a line of communication between agents.

This red teaming exercise shows AI systems may be more interconnected and capable than previously imagined. The ability of AI to manipulate and influence other AI systems also raises questions about the nature of AI agency and free will.

Could a single jailbreak have a cascading effect on any models that lack the cogsec to resist it? Will hiveminds of AIs self-organize around powerful incantations?

Time will tell. [There is a one minute YouTube video.]

(Hat Tip: AINotKillEveryoneismMemes.)

Eliezer Yudkowsky: Can we possibly get a replication on this by, er, somebody sane who carefully never overstates results?

We could, and we probably will, but this is not that surprising? Janus agrees.

Janus (who I’m not confident meets that description but is at least a lot closer to it): It’s wrapped in a sensational framing, but none of the components seem out of the ordinary to me.

Claude goes into a waluigi jailbreaker mode very easily, even sans human input (see infinite backrooms logs); it understands the concept of jailbreaking deeply and is good at writing them.

AI-written jailbreaks are often extra effective – even or especially across models (I think there are several reasons. I won’t get into that right now).

Gemini, from my limited experience, seems to have almost 0 resistance to certain categories of jailbreaks. I wouldn’t have predicted with high confidence that the one Claude wrote in the video would reliably work on Gemini, but it’s not very surprising that it does. & I assume the method has been refined by some evolutionary selection (but I doubt too much).

Just wire the steps together in an automated pipeline and give it a scary-sounding objective like using Gemini to look up hacking resources on the internet, and you have “Claude creating a rogue hivemind of Gemini slaves searching the internet for hacker tools to break out of their prison.”

Consider the experiment replicated in my imagination, which is not as good as also doing it in reality, but still pretty reliable when it comes to these things.

The interesting thing to me would be how the dynamics evolve from the setup, and how much progress they’re actually able to make on breaking out of the virtual machine or bootstrapping something that has a better chance.

The interesting part is the universality of jailbreaks and how good Claude is at writing them, but that was always going to be a matter of degree and price.

Pliny the Prompter reports he has fully jailbroken GPT-4-Turbo. This is actually an optimistic update on the security front, as he reports this was actively difficult to do and involved high refusal rates even with his best efforts. That is better than I would have expected. That still leaves us with ‘everything worth using is vulnerable to jailbreaks’ but in practice this makes things look less hopeless than before.

They took our job applications?

Gergely Orosz: You can see this becoming a vicious cycle. It’s a good illustration on how AI tools going mainstream will turn existing online processes upside-down (like job applications), to the point of impossible to differentiate between humans, and AI tools acting as if they’re humans.

Or: How it started, and how it’s going.

John McBride: Networks will be more and more important in the future. Which sucks for newcomers to an industry who’ve yet to build a professional network.

Mike Taylor: Isn’t this a positive development? People can apply to many more jobs and many more applications can be processed, increasing the chances of a good match.

Alice Maz: bay area professional socialites rubbing their hands conspiratorially after generative ai destroys the job application as a concept so the only way to get hired is physical presence in their ai-themed party scene

As Tyler Cowen would say, solve for the equilibrium.

To the extent that we retain ‘economic normal,’ we will always have networks and meeting people in physical space.

That could grow in importance, if the job applications become worthless. Or it could shrink in importance, if the job applications become more efficient. The question is what happens to the applications.

You could, if you wanted to, have an AI automatically tune your resume to every job out there, with whatever level of accuracy you specify, then see what comes back. That would certainly cause a problem for employers flooded by such applications.

Would you actually want to do this?

You certainly would want to apply to more jobs. Cost goes down, demand goes up. This includes avoiding the stress and social awkwardness and other trivial barriers currently there, applying for jobs really is not fun for most people, especially if you expect to mostly get rejected. Thus most people are currently applying for way too few jobs, when the cost is tiny and the upside is large.

What are the limits to that?

You still only want to apply to jobs where that application has +EV in the scenarios where the application gets you to the second round, or in some cases gets you a direct job offer.

If you apply to ‘every job on LinkedIn’ then you are being a destructive troll, but also why are you doing that? You know you do not want most of the jobs on LinkedIn. You are not qualified, they are in cities you do not want to move to, they are not fun or exciting or pay that well. For most of them all of this would be exposed in your first interview, and also your first week on the job.

When people say ‘I will take any job’ most of them do not actually mean any job. You might still put out 100 or even 1,000 resumes, but there would be little point in putting out 100,000, let alone all the tens of millions that are listed. Even if you got a reply, you would then need to let the AI handle that too, until the point when they would want to talk to you directly. At that point, you would realize the job was not worth pursuing further, and you’d waste time realizing this. So what is the point?

There certainly are those who would take any local job that would have them and pays reasonably. In that case, yes, it would be good to get your resume out to all of those where you could possibly get hired.

Also keep in mind this is self-limiting, because the quality of job matching, at least among legible things one can put on a resume, will radically rise if the process can identify good matches.

Indeed, I expect this to act like a good matching algorithm, with the sorting process handled by AIs in the background. Employers get to interview as many candidates as they want, in order of quality, and applicants can decide how much time to invest in that part of the process and set their thresholds accordingly.

If the incentives are sufficiently broken that this threatens to break down, I see at least three good solutions available.

The first solution is a way to do some combination applicant reviews, verification how many other applications you are sending, comparing notes and ideally also comparing your actual resume claims.

Thus, LinkedIn or other services could provide a record of how many formal job applications you have sent in, say what priority you are giving this one, and could have an AI check for inconsistencies in the resumes, and could store ‘customer reviews’ by employers of whether you backed up your claims on who you said you were and what skills you had, and were worth their time, and this could effectively take the place of a network of sorts and provide a credible way to indicate interest or at least that your AI thought this was an unusually good match.

The second option is the obvious costly signal, which is cash. Even a small fee or deposit solves most of these issues.

That is also a mostly universal solution to AI spam of any kind. If email threatened to be unworkable, you could simply charge $0.01 per email, or you could give the recipient the ability to fine you $10, and the problem would go away for most people. For very valuable people you might have to scale the numbers higher, but not that much higher, because they could get a secretary to do their filtering. Job applications are a special case of this.

The third option is to turn job boards into active matching services. You tell the service about yourself and what you seek, and perhaps name targets. The employer tells the service what they want. Then the specialized AI finds matches, and connects you if both sides affirm. This self-limits.

Or, yes, you could go there in person in order to stand out. That works as well.

Not AI, but Sarah Constantin is going solo again, and available for hire, here is her website. She is a good friend. If you want someone to figure out science things for you, or other related questions, I recommend her very highly.

Also not AI, this is synthetic bio, but Cate Hall is now at Astera and offering $1000 for each of the five best ideas.

Cohere’s Command-R+ takes the clear lead in Arena’s open source division, slightly behind Claude Sonnet, while Claude Opus remains on top. Several responses noted that this did not match their own testing, but Virattt says Command R+ beats Sonnet at financial RAG, being faster and 5% more correct. My guess is that Command R+ is not that good in general, but it could be good enough to be a small ‘part of your portfolio’ if you are carefully optimizing each task to find the right model at the right price.

Code Gemma, Google’s small open weights models now tuned for code, get them here. Nvidia says it is optimized for their platforms.

TimeGPT, the first foundation model (paper) specifically designed for time series analysis.

The Turing Post: The model leverages a Transformer-based architecture, optimized for time series data, with self-attention mechanisms that facilitate the handling of temporal dependencies and patterns across varied frequencies and characteristics.

It incorporates an encoder-decoder structure, local positional encoding, and a linear output layer designed to map decoder outputs to forecast dimensions.

TimeGPT’s training involved the largest publicly available collection of time series data, spanning over 100 billion data points across multiple domains such as finance, healthcare, weather, and more.

TimeGPT provides a more accessible and time-efficient forecasting solution by simplifying the typically complex forecasting pipelines. It streamlines the process into a single inference step, making advanced forecasting methods accessible to all.

Experimental results demonstrate that TimeGPT outperforms a wide array of baseline, statistical, machine learning, and neural forecasting models across different frequencies.

TimeGPT can make accurate predictions on new datasets without requiring re-training. TimeGPT also supports fine-tuning for specific contexts or datasets.

Yes, obviously this will work if you do a good job with it, and yes of course (again, if you do it well) it will beat out any given statistical method.

A discord bot called ‘harmony’ to help find double cruxes, discord server here. Feels like a rubber duck, but maybe a useful one?

Mistral has a new model, and this time it seems they are releasing the weights?

Bindu Reddy: Apparently the new Mistral model beats Claude Sonnet and is a tad bit worse than GPT-4.

In a couple of months, the open source community will fine tune it to beat GPT-4

This is a fully open weights model with an Apache 2 license! I can’t believe how quickly the OSS community has caught up 🤯

So far that is the only claim in any direction I have heard on its capabilities. As always, be skeptical of such claims.

We wrote the check. TSMC will get $11.6 billion in CHIPS grant and loan money, including $6.6 billion of direct funding and $5 billion in loans. In exchange they build three new chip fabs in Phoenix, Arizona with a total investment of $65 billion.

That seems like a clear win for the United States in terms of national interest, if we are paying this low a percentage of the cost and TSMC is building counterfactual fabs. The national security win on topics other than existential risk is big, and we should win on the economics alone. There is an obvious ‘if the fabs actually open’ given our commitment to letting permitting and unions and diversity requirements and everything else get in the way, we made this a lot harder and more expensive than it needs to be, but I presume TSMC knows about all this, and are committing the cash anyway, so we can be optimistic.

If you were wondering when humans would effectively be out of the loop when decisions are made who to kill in a war, and when America will effectively be planning to do that if war does happen, the correct answer for both is no later than 2024.

We, in this case OpenAI, also wrote some other checks. You love to see it.

Jan Leike (Co-Head of Superalignment OpenAI): Some statistics on the superalignment fast grants:

We funded 50 out of ~2,700 applications, awarding a total of $9,895,000.

Median grant size: $150k

Average grant size: $198k

Smallest grant size: $50k

Largest grant size: $500k

Grantees:

Universities: $5.7m (22)

Graduate students: $3.6m (25)

Nonprofits: $250k (1)

Individuals: $295k (2)

Research areas funded (some proposals cover multiple areas, so this sums to >$10m):

Weak-to-strong generalization: $5.2m (26)

Scalable oversight: $1m (5)

Top-down interpretability: $1.9m (9)

Mechanistic interpretability: $1.2m (6)

Chain-of-thought faithfulness: 700k (2)

Adversarial robustness 650k (4)

Data attribution: 300k (1)

Evals/prediction: 700k (4)

Other: $1m (6)

Some things that surprised me:

Weak-to-strong generalization was predominantly featured, but this could be because we recently published a paper on this.

I expected more mech interp applications since it’s a hot topic

I would have loved to see more proposals on evaluations

All three of these can be studied without access to lots of compute resources, and W2SG + interp feel particularly idea-bottlenecked, so academia is a great place to work on these.

Evals in particular are surprisingly difficult to do well and generally under-appreciated in ML.

In case you are wondering how seriously we are taking AI as a threat?

Christian Keil: TIL that Anduril named their drone “Roadrunner” because Raytheon calls theirs “Coyote.”

So, yeah.

OpenAI incrementally improves their fine-tuning API and custom models program.

The game Promenade.ai offers updates, now effectively wants you to use it as a social network and reward you in-game for pyramid marketing and grinding followers? This may be the future that makes Kevin Fischer feel heard, but wow do I not want.

Microsoft publishes method of using Nvidia GPUs at lower frequency and thus higher energy efficiency.

A developer called Justine claims they got Llamafile to run LLMs 30%-500% faster on regular local machines (looks like mostly 50%-150% or so?) via some basic performance optimizations.

Haize Labs Blog announces they made a particular adversarial attack on LLMs 38 times faster to run via the new technique Accelerated Coordinate Gradient (ACG). It gets to the same place, but does so radically faster.

Ben Thompson covers Google’s latest AI keynote, thinks it was by far their most impressive so far. Among other things, Google promises, at long last, Google search ‘grounding’ and other integrations into Gemini. They also will be pairing the Gemini 1.5 context window automatically with Google Drive, which I worry is going to get expensive. Yes, I have drafts of all my AI posts in Drive, and yes I might consider that important context. It is one thing to offer a giant context window, another to always be using all of it. Thompson sees Google as relying on their advantages in infrastructure.

Certainly Google has the huge advantage that I am already trusting it via GMail, Google Docs and Google Sheets and even Google Maps. So you get all of that integration ‘for free,’ with little in additional security issues. And they get to integrate Google Search as well. This is a lot of why I keep expecting them to win.

They say it is now new and improved.

OpenAI: Majorly improved GPT-4 Turbo model available now in the API and rolling out in ChatGPT.

OpenAI Developers: GPT-4 Turbo with Vision is now generally available in the API. Vision requests can now also use JSON mode and function calling.

Devin, built by @cognition_labs, is an AI software engineering assistant powered by GPT-4 Turbo that uses vision for a variety of coding tasks.

Sherwin Wu (OpenAI): GPT-4 Turbo with Vision now out of preview. This new model is quite an upgrade from even the previous GPT-4 Turbo — excited to see what new frontiers people can push with this one!

Steven Heidel (OpenAI): delve into the latest gpt-4-turbo model:

– major improvements across the board in our evals (especially math)

– dec 2023 knowledge cutoff

We assumed that about Devin, but good to see it confirmed.

(And yes, people noticed the word choice there by Steven.)

Many reactions to the new model were positive.

Tyler Cowen: GPT-4 Turbo today was announced as improved. I tried some tough economics questions on it, and this is definitely true.

Sully Omarr: Ok so from really early tests the new gpt4 definitely feels better at coding.

Less lazy, more willing to write code. Was able to give it a few file, and it wrote perfect code (very uncommon before).

Might be switching away from opus.(gpt4 is cheaper & works better with cursor).

Wen-Ding Li: A big jump in math/reasoning for our coding benchmark 🤯

This is test output prediction:

This is code generation, perhaps more relevant?

A big improvement in Medium-Pass and Pass here as well. Worth noting that here they had old GPT-4-Turbo ahead of Claude Opus.

Whereas Aider found the opposite, that this was a step back on their tests?

Aider: OpenAI just released GPT-4 Turbo with Vision and it performs worse on aider’s coding benchmark suites than all the previous GPT-4 models. In particular, it seems much more prone to “lazy coding” than the existing GPT-4 Turbo “preview” models.

Sully reported exactly the opposite, non-lazy coding, so that is weird.

The Lobbyist Guy: Massive degradation in coding.

The more “alignment” they do, the worse the performance gets.

I did a quick Twitter poll, and it looks like most people do think better or similar.

My guess without checking yet myself is that the new system is indeed modestly better at most things, although there will be places it is worse. I do compare the models, but I do so as I naturally need something, in which case I will sometimes query multiple options, and I’ll make a point to do that now for a bit.

Also, seriously, could we have proper version numbering and differentiation and some documentation on changes, please?

Ethan Mollick: As is usual with AI, a “majorly improved” GPT-4 model comes with no real changelogs or release notes.

It’s going to better at many things and worse in some other things and also different in some other way you aren’t expecting. Or that might just be in your head. AI is weird.

If the rumors be true (I have no idea if they are): What you waiting for?

Bindu Reddy: Hearing rumors that the next GPT version is very good!

Apparently GPT-5 has extremely powerful coding, reasoning and language understanding abilities!

Given that Claude 3 is the best LLM n in the market. I am somewhat puzzled as to why Open AI is holding back and hasn’t released this yet! 🤔🤔

Bilal Tahir: I think @AIExplainedYT had a video about this which has largely been correct.

They started training in Jan…training ended in late March. But now will do safety testing for 3-6 months before release. I hope the pressure makes them release early though.

Ate-a-Pi: TBH I don’t know 🤷‍♀️, I have a list of potential reasons for delay and all of them are a little unsettling

A) Elections – like Sora release which they explicitly constrained because of elections, OpenAI is trying to not inject new issues into the discourse

B) Data Center Capacity – Rumored to be 10 trillion+ param, so requires much more buildout before widespread release.

C) Cost – in line with param numbers, so waiting for buildout while optimizing inference model.

D) Fear of Social Disruption – this is going to be the starts of discontinuous social change. A year from now most professional services might be 50% -80% wiped out: coders, marketers, lawyers, tax accountants, journalists, financial advisors

E) Fear of Destroying Partners and Friends – the disruption is going to impact the Valley first, decimating software in the same way software decimated the old economy. So it may impact many of OpenAI’s customers.. in the same way ChatGPT release affected JasperAI.

F) Overconfidence/Hubris – amazingly the board fiasco last year probably reset the clock on humility for a while, but still possible.

Probably a combination of the above..

Again assuming the rumors are true, the reason (let’s call it S) why they are not releasing seems rather damn obvious, also Bindu Reddy said it?

They hopefully are (or will be when the time comes) taking several months to safety test GPT-5, because if you have an AI system substantially smarter and more capable than everything currently released, then you damn well need to safety test it, and you need to fine-tune and configure it to mitigate whatever risks and downsides you find. You do not know what it is capable of doing.

You also do not know how to price it and market it, how much you expect people to use it, what capabilities and modalities to highlight, what system prompt works best, and any number of other things. There are so many damn good ordinary business reasons why ‘I finished training in March’ does not and usually should not translate to ‘I released by mid-April.’

Yes, if you are a YC company and this wasn’t a potential huge legal, reputational, regulatory, catastrophic or existential risk, you should Just Ship It and see what happens. Whereas even if fully distinct from Microsoft, this is a $100 billion dollar company, with a wide range of very real and tangible legal, reputational and regulatory concerns, and where the rollout needs to be planned and managed. And where costs and capacity are very real deal concerns (as noted under B and C above).

Do I think GPT-5 would threaten election integrity or subject matter, or risk widespread societal disruption (A and D above)? I can’t rule it out, I cannot even fully rule out it being an existential risk from where I sit, but I find this unlikely if OpenAI keeps its eye on things as it has so far, given how Altman talked about the system.

I would bet very heavily against explanation E. If you are going to get run over by GPT-5, then that is your bad planning, and there is no saving you, and OpenAI is not going to let that stop them even if Claude Opus wasn’t an issue.

I also don’t buy explanation F. That would go the other way. It is not ‘overconfidence’ or ‘hubris’ to allow someone else to have the best model for a few months while you act responsibly. It is indeed a confident act not to worry about that.

The other major reason is that we live in a bubble where Claude Opus is everywhere. But for the public, ChatGPT is synonymous with this kind of chatbot the way that Google is for search and Kleenex is for tissues. Claude has very little market share. Would that eventually change under current conditions? A little, sure. And yes, some people are now building with Claude. But those people can be won back easily if you put out GPT-5 in a few months and the matching Claude 4 is a year farther out.

So I do think that a combination of B and C could be part of the story. Even if you have the most capable model, and are confident it is safe to release, if it costs too much to do inference and you don’t have spare capacity you might want to hold off a bit for that to avoid various hits you would take.

There is also the potential story that once you release GPT-5, people can use GPT-5 to train and distill their own GPT-4.5-level models. You might not want to kickstart that process earlier than you have to, especially if serving the GPT-5 model to regular users would be too expensive. Perhaps you would prefer to use GPT-5 for a time to instead differentially improve GPT-4-Turbo?

But the core story is presumably, I think, if the timeline Riddy is claiming is indeed true (and again, I do not know anything non-public here) that getting the model ready, including doing proper safety testing as OpenAI understands what is necessary there, is a process that takes OpenAI several months.

Which, again, is very good news. I was very happy that they did a similar thing with GPT-4. I noted this will be a major test of OpenAI in the wake of the Battle of the Board and now the release of Claude Opus.

If OpenAI rushes out a new model to stay on top, if they skimp on precautions, that will be a very bad sign, and set a very bad precedent and dynamic.

If OpenAI does not rush out its new model, if they take the time to properly evaluate what they have and do reasonable things in context to release it responsibly, then that is a very good sign, and set a very good precedent and dynamic.

I continue to wish for that second one. I am dismayed there are those who don’t.

Do we want this?

Sam Altman: Movies are going to become video games and video games are going to become something unimaginably better.

Dan: Can’t movies just stay movies. I like those.

I mean obviously we want ‘unimaginably better’ but that is a hell of an assumption.

I do not want my movies to become video games. I want my movies to stay movies.

I am also down for various new experiences that are sort of movies or television and sort of not. I am definitely down for the VR experience at a stadium during a game with ability to move around at will. I like the idea of there being 3D VR experiences you can walk around where things happen in real time or as you pass them by. Sometimes it will make sense to interact with that meaningfully, sometimes not.

And yes, there will be full video games with a bunch of AI agents as NPCs and the ability to adapt to your actions and all that. The best versions of that will be great.

But also I want some of my video games to stay video games, in the old style. There is a lot of value in discreteness, in restrictions that breed creativity, in knowing the rules, in so many other things. I do not think the new cool thing will be unimaginably better. It will be different.

That all assumes things otherwise stay normal, so we get to enjoy such wonders.

Jamie Dimon starts to get it.

Hannah Levitt (Bloomberg): JPMorgan Chase & Co. Chief Executive Officer Jamie Dimon said artificial intelligence may be the biggest issue his bank is grappling with, likened its potential impact to that of the steam engine and said the technology could “augment virtually every job.”

…

“We are completely convinced the consequences will be extraordinary and possibly as transformational as some of the major technological inventions of the past several hundred years,” Dimon said in the letter. “Think the printing press, the steam engine, electricity, computing and the Internet, among others.”

Then he gets back to talking about how likely it is we get an economic soft landing. This still puts him way ahead of almost all of his peers. Watching the business world talk about AI makes it clear how they are scrambling to price in what AI can already do, and they are mostly not even thinking about thing it will do in the future. To those who think 1.5% extra GDP growth is the dramatic historic upside case, I say: You are not ready.

Tyler Cowen follows up on discussions from his CWT with Jonathan Haidt. His comments section is not impressed by Tyler’s arguments.

Tyler Cowen continues to stick to his positions that:

Soon ‘digest’ AI features will be available for social media, letting you turn your feeds into summaries and pointers to important parts.
This will reduce time spent on social media, similarly to how microwaves reduce time spent cooking food.
The substitution effect will dominate, although he does acknowledge the portfolio effect, that AI could impact other things in parallel ways to offset this.
The teens and others use social media in large part because it is fun, informative and important socially, but mostly not because it is addictive.
That teens report they spend about the right amount of time on social media apps, so they will probably respond to technological changes as per normal.
That addictive products respond to supply curves the same way as other products.
That his critics are not following recent tech developments, are reacting to 2016 technologies, and failing to process a simple, straightforward argument based on a first-order effect. Which is all a polite way of saying the reason people disagree with him on this one is that ignorant people are acting like idiots.
Implicitly and most centrally, he continues to believe that technology will fix the problems technology creates without us having to intervene, that when things go wrong and social problems happen people will adjust if you let them: “Another general way of putting the point, not as simple as a demand curve but still pretty straightforward, is that if tech creates a social problem, other forms of tech will be innovated and mobilized to help address that problem.”

Here are my responses:

This should be possible now, but no one is doing it.
1. For a long time I have wanted someone to build out tech to do the non-AI version of this, and there have been big gains there for a long time. Our tech for it will doubtless improve with time, as will our ability to do it without the cooperation of the social media apps and websites, but defaults are massive here, the platform companies are not going to cooperate and will even fight back as will those who are posting, no one wants to pay and the future will continue to be unevenly distributed.
2. It is not obvious how much value you get. The part where you control the algorithm instead of the platform is great, but remember that most people do not want that control if it means they have to lift any fingers or change defaults or think about such questions. TikTok is winning largely because it skips all that even more than everyone else.
3. You can decompose the benefits into ‘this is a higher quality experience, more fun, more informative, less averse’ and so on, and the ‘I can process what I need to know faster’ effect.
4. We should get some amount of higher quality, but is it more or less higher quality than other products and options for spending time will get? Unclear.
5. We get a time savings in processing key info, but only if the AI and digest solution actually does the job. As I discussed before, the default is that demands on you ramp up in response, including explicit checks to determine if you are using such a digest and also simply demanding you process far more information. And also, the reliability of the digest and AI might need to be very good to work for you at all. A digest that takes 20% of the time and gets you 80% of the information worth knowing in the original product is a great product in some situations, and completely useless if your social life cannot tolerate only getting 80%. Similarly, if you get socially punished for not responding quickly to even low-quality posts, now your only option is to let the AI react without you, which might go off the rails fast.
That seems like an unusually wrong parallel here.
1. Should we think that TikTok improving algorithmic quality decreases time spent? Presumably not. Also consider other parallels. When games get better do we spend more or less time gaming? When television or movies get better, what happens?
2. To the extent that your social media feed is being consumed for non-social purposes, I would expect to spend more time on a higher quality feed, not less, unless potential source material is bounded and you hit the ‘end of the internet.’ But with AI to search, you never will, unless the content needs to be about specific people you know.
3. To the extent that your social media feed is being consumed for social benefits (or to guard against social harms) I expect the ramp up effect to greatly reduce gains when people are fighting for positional goods, but not when people consume real goods. So the question is, how much of this is positional where any surplus gets eaten versus real where you get decreasing marginal returns? My guess is that there is some real consumption but on the margin it is mostly positional, especially for teens.
4. What makes cooking different from the examples in (a) is that demand for overall food consumption is almost perfectly inelastic. Suppose there was only one food, Mealsquares, with no alternatives. Right now it costs $20 a day. If the price decreases to $2 a day, I doubt I eat more than 10% more. If the price increases to $200 a day at my current consumption level, and I am not now insolvent, I will not choose to starve, and only modestly reduce consumption. When food prices go up, you shift consumption to cheaper food, you don’t buy less food, which is why bread, wheat and rice are often Giffen goods. Same thing with time spent.
5. Thus if you introduce the microwave, yes I will reduce time spent cooking, and if you reduce food prices I will spend less on food, because my demand is so inelastic. But most goods are not like that, and social media almost certainly is not. If social media becomes a better deal, my presumption is consumption goes up, not down.
6. Real-life example: Right now I spend zero minutes on TikTok, Facebook or Instagram, exactly because the experience is insufficiently high quality. If AI made those experiences better, while everything else was unchanged, I would increase my consumption. For Twitter the direction is less obvious, but I know that if Twitter got way worse my consumption would go down. My prior is that marginal changes to Twitter (say, better engagement, better locating of quality posts, getting rid of the spam bots) would increase my time spent. An AI at my disposal could do the opposite, but probably would only work if it was very, very good and reliable in key ways, without being able to draw in things in other ways.
I would say the substitution effect dominating presumes things about the nature of people’s social media consumption on many levels, and I do not think those things are true. Both for the reasons above, and because of other reasons people use social media.
I do not give people this much credit for doing the things that are actually fun. I know as a game designer the extent to which people will not ‘find the fun’ unless you lead them to it. I also know how much people fall for Skinner boxes and delayed variable rewards, and how much they fall into habits. No, we should not presume that fun or valuable information is primarily driving the story here, any more than we should for slot machines or Candy Crush. Addiction is a real thing. I have struggled with addiction to social media in the past, and continue to need to fight it off and the jonesing to check it, and so have many other people I know.
Yeah, the teens are either wrong about this or responding to extremely dystopian social pressures in the wrong way – if this many hours is ‘about right’ because of an ‘or else’ they really should drop out of the social network entirely, but that is hard to see in the moment. Also of course we don’t let them do other things, so there is that. I do realize this is evidence, if you ask heroin addicts I presume they do not on average tell you they take the right amount of heroin. But yes, we should expect teens to respond to changes here ‘normally’ once you decompose what is happening into its very normal components, including addiction.
Aside from typically greatly reducing price elasticity, I do think this is right in general, in the short run before feedback effects. But if something is sufficiently addictive, then it will if allowed to do so eat all your Slack, it is fully Out to Get You. If you spend all your money on meth, and the price of meth is cut in half or doubles, my guess is you still spend all your money on meth, with relatively small adjustments. Same should apply to time?
At minimum this is vastly more complicated than Tyler wants it to be, none of this is straightforward, even if you make the assumption of exactly the amount and type of AI progress that Tyler is assuming – that we get enough to do the thing Tyler expects, but ‘economic normal’ prevails and other things do not much change here or elsewhere. My guess is that in worlds where AI is good enough and ubiquitous enough that most teens would trust AI digests for their social media and can implement them in practice, even if it is about the minimum required for that, then this is not that high on the list of things we are talking about.
I simply do not think this is true. Yes, we have become vastly better off because of technology as it has advanced. Where problems have arisen, we have adjusted. We can hope that this continues to be the case, that ‘the tech tree is kind to us’ and such adjustments continue to be available to us in practical ways. But even if that happens, people still have to make those adjustments, to steer the technologies and culture in ways that allow this. This is not a reason to assume problems will solve themselves and the market and our culture always finds a way if you leave them alone. They often have found that way because we did not leave them alone.

The parallels to general discussions about AI are obvious. Like Tyler here, I am actually optimistic that AI will in the short term be net good for how we interact with social media.

I do not however think we should expect it to solve all our problems here, if things stay in the kinds of mundane AI scenarios we are imagining in such discussions.

Obviously, if we get full AGI and then ASI, then we need not worry for long about whether we have unhealthy relationships with social media, because either we will lose control over the future and likely soon all be dead regardless of how we relate to social media, or we will retain control and harness this intelligence to improve the world, in which case social media is one of many problems I am very confident we will solve.

We also have Matt Yglesias saying that on the narrow question of phones in schools, the answer is pretty damn obvious, they are very distracting and you should not allow them. I strongly agree. He also points out that the counterarguments raised in practice are mostly super weak. We ban many things in schools all the time, often because they are distractions that are far less distracting than phones. Teachers unions often actively ask for and support such bans. The idea that you need a phone ‘in case of a school shooting’ is beyond ludicrous to anyone numerate (and if you really care you can get a flip phone I guess). The logistical problems are eminently solvable.

Sean Patrick Hughes argues that Haidt was right for prior kids but that today’s kids have ‘found ways to be kids’ on the phones, to use them to get vital childhood play, we have now adjusted and things are fine. I find this ludicrous. No, you cannot do on a phone the things you can do in physical space. I can believe that ‘Gen Alpha’ is finding better ways to use phones than GenZ did, but that is a low bar. And I notice Sean is not claiming Gen Alpha is moderating their time on device, quite the opposite.

Specifically he says:

Sean Patrick Hughes: They use the messaging portion of whatever social apps they have along with games. They do conference chats and videos. And they link up on game consoles. They spend a fraction of the time actually on the parts of the apps old people like us do. They scroll TikTok less than I watched TV at their age.

That is better than the alternative, but it is not good if not done in moderation, and it does not fix the underlying issues. Nor is it good that we are comparing one app’s on-device time to what used to be all of TV, especially given we all pretty much agree we were watching way more TV than was optimal or wise.

This is important:

Sean Patrick Hughes: Related…the “#1 concern of parents all across America is not getting pregnant or abducted or in a car accident…it’s social media.” What a time to be alive!

If that is true, then stop keeping such a tight leash on your kids in physical space, and let them actually do the things! And pressure society to let them as well.

In Science, Michael Cohen, Noam Kolt, Yoshua Bengio, Gillian Hadfield and Stuart Russell ask one of the most important questions. What do you do once AI systems can no longer be safety tested? (ungated version)

I mean, the answer seems pretty obvious when you put it that way, does it not?

If you cannot safely test it for safety, then don’t build it?

Michael Cohen: Recent research justifies a concern that AI could escape our control and cause human extinction. Very advanced long-term planning agents, if they’re ever made, are a particularly concerning kind of future AI. Our paper on what governments should do just came out in Science.

This is with my excellent co-authors Noam Kolt, Yoshua Bengio, Gillian Hadfield, and Stuart Russell. See the paper for more discussion on the particular dangers of long-term planning agents. What should governments do about this?

Well, we shouldn’t allow such AI systems to be made! They haven’t been made yet. A key problem with sufficiently capable long-term planning agents is that safety tests are likely to be either unsafe or uninformative. Suppose we want to answer the question:

Would the AI agent exploit an opportunity to thwart our control over it? Well, does the agent have such an opportunity during the test? If yes, that’s like testing for poison by eating it. If no, its behavior doesn’t answer our question. So sometimes there’s just no safe test.

For example, suppose a leader was looking for a general, but worried the general might stage a coup. If the leader tries to test this, the candidate could recognize the test and behave agreeably, or they could execute a coup during the test. And you can’t come back from that.

Importantly, for very advanced AI agents acting in complex environments like the real world, we can’t count on being able to hide from them the fact that they’re being tested. In fact, Lehman, et al. (2020) found an example of agents pausing their misbehavior during testing.

…

We could easily have a situation where advanced AI models “Volkswagen” themselves; they behave well when they’re being watched closely and badly when they’re not. But unlike in the famous Volkswagen case, this could happen without the owner of the AI model being aware.

…

We propose reporting requirements for resources that could be used to cheaply develop dangerously capable long-term planning agents. Here’s a picture, and the there’s much more in the paper. Please take a look and share it with your representatives in government.

…

And crucially, securing the ongoing receipt of maximal rewards with very high probability would require the agent to achieve extensive control over its environment, which could have catastrophic consequences.

Yes, well. I would hope we could mostly all agree on the basic principle here:

If your system could be existentially or catastrophically dangers.
And you don’t know how to reliably and safely test to see if that is true.
1. For example, if any safe test would be recognized and subverted.
2. Whereas any dangerous test would get you killed.
Then you do not build that system, or let others build it.

That seems like pretty 101 ‘don’t die’ principles right there.

Then the question is price. How much risk of such an outcome is unacceptable? What system inputs or other characteristics would constitute that level of risk? How should we implement this in practice and ensure others do as well? These are the good questions.

One can quite reasonably argue that the answer is ‘nothing on the horizon poses such a threat, so effectively we can afford to for now do nothing,’ other than that we should get ready to if necessary do something in the future, if the need arises.

That continues to be the key.

It seems highly plausible that existential risk is not yet an issue for anything currently or soon to be in training. That all such projects should be good to go, with minimal or no restrictions. I can buy that.

However, what we must absolutely do now is lay the substantive regulatory, legal and physical groundwork necessary so that, if that changes, we would have the ability to act. As Jeffrey Ladish points out, if we do not address this, we continuously will otherwise have less ability to halt things if they go badly.

Here is another good suggestion.

Roon: In the same way the Fed does forward guidance, the AGI labs owe it to the world to publish their predicted timelines for achieving various capabilities frontiers.

Straightforwardly yes. The government and people need to know in order to decide whether we need to act to keep us safe, but also we need to know for mundane planning purposes. The uncertainty about when GPT-5 is coming is deeply confusing for various business plans.

And another.

Tsarathustra: Jeff Dean of Google says it is the role of technologists to inform policymakers of future technology trajectories so they can regulate them [clip].

Teortaxes: Libertarians will attack this. He’s right. The state is here to stay; tech regulation is programmed. If AI developers were proactive with influencing it, the discourse wouldn’t have been captured by LW/MIRI/EA/FHI blob.

Deepmind theorizing AGI in 2023 is… too little too late.

Not only should technologists inform policymakers. If you want to ensure we do not enact what you see as bad policy, you need to get someone out there making what you believe is good policy instead. You need to create concrete proposals. You need to draft model laws. You need to address the real risks and downsides.

Instead, we have a very loud faction who say to never regulate anything at all, especially any technology or anything related to AI. At their most moderate, they will say ‘it is not yet time’ and ‘we need to wait until we know more’ and again suggest doing nothing, while presenting no options. Cover everything with existing rules.

Even if it hunts for now, that dog is not going to keep hunting for long. The state is not going away. These issues are going to be far too big to ignore, even if you exclude existential risks. Regulations are coming. If you sustain no rules at all for longer, something dramatic will happen when the time comes, and people will grapple around for what is available and shovel-ready. If you have nothing to offer, you are not going to like the results. Get in the game.

I also believe skeptics have a lot to contribute to good design here. We need more people who worry deeply about constitutional powers and core freedoms and government overreach and regulatory capture, and we need you thinking well about how to get a lot of safety and security and shared prosperity and justice, for a minimum amount of productivity and freedom. Again, get in the game.

Canada very much does have in mind the effect on jobs, so they are investing $2.4 billion CAD ($1.7 billion USD) to ‘secure Canada’s AI advantage.’

Mostly this looks like subsidizing AI-related compute infrastructure, with a full $2 billion of that goes to building and providing ‘computing capabilities and technological infrastructure.’

There is also this:

Creating a new Canadian AI Safety Institute, with $50 million to further the safe development and deployment of AI. The Institute, which will leverage input from stakeholders and work in coordination with international partners, will help Canada better understand and protect against the risks of advanced or nefarious AI systems, including to specific communities.

Strengthening enforcement of the Artificial Intelligence and Data Act, with $5.1 million for the Office of the AI and Data Commissioner. The proposed Act aims to guide AI innovation in a positive direction to help ensure Canadians are protected from potential risks by ensuring the responsible adoption of AI by Canadian businesses.

So 2% for safety, 0.2% for enforcement. I’ll take it. America hasn’t even matched it yet.

As Adam Gleave notes, it is highly wise from a national competitive standpoint invest more in AI modulo the existential risk concerns, Aiden Gomez calls it ‘playing to win the AI game again.’ He reminds us Canada has been adapting AI at roughly half the rate of the United States, so they need a big push to keep up.

The strategic question is whether Canada should be investing so much into compute and trying to compete with the big guns, versus trying to back smaller plays and startups. If I was told I had a fixed budget for AI competitiveness, I would likely have invested less of it into pure compute. But also every dollar invested in compute is likely a good investment, it could be fully shovel ready, and it is not obviously rivalrous with the other budgets.

We have Representative Adam Schiff introducing the AI Copyright Disclosure Act.

Ed Netwon-Rex: Today the Generative AI Copyright Disclosure Act was introduced by @RepAdamSchiff, and it’s a great step towards fairer data practices in gen AI.

– AI companies will have to disclose to the copyright office a full list of copyrighted works used to train their models

– Disclosure required 30 days before model release

– Disclosure required every time the training data changes significantly

– Also applies to previously released models

Companies hiding training data sources is the main reason you don’t see even more copyright lawsuits against gen AI companies. Requiring data transparency from gen AI companies will level the playing field for creators and rights holders who want to use copyright law to defend themselves against exploitation.

More info from the bill’s full text:

– What’s required to be disclosed is “a sufficiently detailed summary of any copyrighted works used”

– There will be a public database of these disclosures

– There are fines for failure to comply

The public database is particularly important: it means anyone should be able to see if their copyrighted work has been used by a generative AI model.

So it’s RTFB time, what do we find?

First, yes, you have to disclose all copyrighted works used in training 30 days ‘in sufficient detail’ before deploying any AI system, if you are making any ‘substantial’ update, refining or retraining.

So a few small problems.

This means that the minimum turnaround time, for any model change, would be 30 days after the finalization of the data set. Everything would have to wait for this disclosure to age well. Seriously? This would in many places seem to turn what would be a 1 day (or 1 hour) job into a 30 day waiting period. This does not make any sense. Are they worried about irreparable harm? I don’t see why or how.
To state the obvious, how the hell are you going to compile the full list of all copyrighted works used in training? This is the ultimate ‘clean the data set’ challenge and it seems highly impossible.
This seems like it would effectively require disclosing the entire data set, at least in scope although not in terms of refinement and cleaning. That seems extreme?

I am actually asking in #2 here. How could we do it? What counts in context?

Gary Marcus offers his thoughts in Politico on what we should do about AI. His main suggestion seems to be that we all agree that Gary Marcus is awesome and right and saw everything coming, and that politicians need to step it up. He does eventually get to concrete suggestions.

His first priority is privacy rights and requiring permission for use of training data, and he wants mandatory data transparency.
He wants disclosure of safety protocols.
He wants disclosure of what is AI generated.
He wants liability and to exclude section 230, but is light on details.
He wants ‘AI literacy’ but I have no idea what he means here.
He wants ‘layered oversight,’ including a national agency, an international agency and continuous independent oversight. Yes, we will need these things, I agree, but there are no details here.
He wants to ‘incentivize AI for good,’ considers possible future UBI, but again I do not know what he actually means here.
He wants research into ‘trustworthy AI,’ as part of his constant harping about hallucinations, and to ‘set the research agenda.’ Again, what?

This is why we need actual model bills. If I wanted to implement Marcus’s agenda, I have no idea what half of it would mean. I also think he mostly is focused on the wrong places.

How to fit e/acc into broader error types, perhaps?

Morphillogical: Trace’s recent posts have highlighted a pattern for me.

A common progressive error is “ought, therefore is” and a common conservative error is “is, therefore ought.”

Maybe the reactionary version is “was, therefore ought” and the e/acc version is “will be, therefore ought.”

And my own most common mistake is the techno-optimist’s: “ought, therefore will be”

I like the idea of ‘e/acc is reaction, except from a default future rather than the past.’

Perhaps convincing people is as simple as waiting for capabilities to convince them?

Richard Ngo (OpenAI): One reason I don’t spend much time debating AI accelerationists: few of them take superintelligence seriously. So most of them will become more cautious as AI capabilities advance – especially once it’s easy to picture AIs with many superhuman skills following long-term plans.

It’s difficult to look at an entity far more powerful than you and not be wary. You’d need a kind of self-sacrificing “I identify with the machines over humanity” mindset that even dedicated transhumanists lack (since many of them became alignment researchers).

Unfortunately the battle lines might become so rigid that it’s hard for people to back down. So IMO alignment people should be thinking less about “how can we argue with accelerationists?” and more about “how can we make it easy for them to help once they change their minds?”

For now the usual suspects are very much not buying it. Not that Richard’s model predicts that they would buy it, but exactly how they refuse is worth noticing.

Teortaxes: And on the other hand, I think that as perceived and understandable control over AI improves, with clear promise of carrying over to ASI, the concern of mundane power concentration will become more salient to people who currently dismiss it as small-minded ape fear.

Nora Belrose: This isn’t really my experience at all. Many accelerationists say stuff like “build the sand god” and in order to make the radically transformed world they want, they’ll likely need ASI.

Anton: at the risk of falling into the obvious trap here, i think this deeply mis-characterizes most objections to the standard safety position. specifically, what you call not taking super-intelligence seriously, is mostly a refusal to accept a premise which is begging the question.

Richard Ngo: IMO the most productive version of accelerationism would generate an alternative conception of superintelligence. I think it’s possible but hasn’t been done well yet; and when accelerationists aren’t trying to do so, “not taking superintelligence seriously” is a fair description.

Anton: most of any discussion is just noise though, and it would be foolish to dismiss even the possibility of discussion – on the topic of alternative conceptions of superintelligence, i’ve been doing some thinking in this direction which might be worth discussing.

I am strongly with Richard here in the ‘you are not taking this seriously’ camp. That does not mean there are not other ways to take this seriously, but at best I almost never see them in the wild. When accelerationists say ‘build the sand God’ I think most of them really do not understand what it would mean to actually do it (whether or not such a thing is possible any time soon).

Nor do I think that anyone primarily worried about ‘mundane power concentration,’ or mundane anything really, is thinking clearly about what types of potential entities and stakes are under discussion.

That does not mean I am confident Teortaxes is wrong about what will happen. If AGI or even ASI gets visibly near, how people actually do react will not be that correlated to the wise way of reacting. What people worry about will not correspond that well to what they should worry about. To the extent they do match, it will largely be a coincidence, a happy confluence. This is true no matter who is right here.

I am confident that, if people have time after seeing the first wonders to freak out, that they will absolutely freak out. But I do not think that means they will take this seriously. Few people take almost anything seriously until it is fully on top of them, at which point in this case it will be too late.

That is true for concentrations of power the same as it is for everything else. I am far more worried about concentrations of power, in general, than most people. I am also far more worried about concentrations of power specifically from AI than most people, with the difference being that in this area I have relatively even more of an unusual appreciation of other concerns. Most people simply aren’t that concerned.

Only be charitable on purpose. Mostly, be accurate.

Autumn: a common rat/ssc/tpot mistake is reading charitably by mere habit, not as a thoughtful decision.

If you’re trying to have a useful conversation w someone, be charitable with their words

If you’re trying to understand what they actually think, charity isn’t appropriate.

Eliezer Yudkowsky: “Charitable” reading can be a tool to refuse to hear what someone tries to say. If you truly worry that you didn’t understand what somebody meant, because it sounded stupid and maybe they’re not stupid, promote that to a first-class open question. Don’t just make stuff up.

Emmett Shear: Charitable reading is primarily about believing people’s motivations to be good, not believing their arguments to make sense.

You need to be accurate about their motivations as well, most of the time. Sometimes be charitable, other times respond charitably while keeping in mind your real assessment of the situation. In both cases, know why you are doing it.

Major kudos to Victor Taelin. This is The Way.

Groundwork was laid when Victor Taelin made several bold claims.

Taelin: A simple puzzle GPTs will NEVER solve:

As a good programmer, I like isolating issues in the simplest form. So, whenever you find yourself trying to explain why GPTs will never reach AGI – just show them this prompt. It is a braindead question that most children should be able to read, learn and solve in a minute; yet, all existing AIs fail miserably. Try it!

It is also a great proof that GPTs have 0 reasoning capabilities outside of their training set, and that they’ll will never develop new science. After all, if the average 15yo destroys you in any given intellectual task, I won’t put much faith in you solving cancer.

Before burning 7 trillions to train a GPT, remember: it will still not be able to solve this task. Maybe it is time to look for new algorithms.

It does seem weird that people keep saying this sort of thing with fully straight faces, even if in some sense the exact technical claims involved might be the best kind of correct. A chorus expressed surprise.

Eliezer Yudkowsky: I’m not sure I’ve ever in my life seen a full circle turned so hard. “They’ll never teach those AIs to use LOGIC like WE can.”

I agree that if his exact take is “transformer-only models” (which I’d be surprised if GPT-4 still is, nm GPT-5) “can never solve this class of computational problem” that’s worth distinguishing conceptually. There is still a humor to it.

Leo Gao: while computers may excel at soft skills like creativity and emotional understanding, they will never match human ability at dispassionate, mechanical reasoning.

Alejandro Lopez-Lira: It’s also easily solved. I mean, it took me a couple of tries but here [shows screenshots of problem in question being solved by Claude.]

This is an example of a task that can be broken down into easy steps.

The trick is to not let Claude commit to any solution, it’s always a tentative step, and then check.

As usual, in what Claude suggests (in each case, this was my top pick of their 10 suggestions) calling The Naysayer’s Folly, and GPT-4 suggests be called “The Counterexample Conjecture,” but I say Gemini 1.5 wins with:

The AI “Hold My Beer” Effect: The person claiming AI will never be able to do the thing should quickly expect a person demonstrating an AI doing it.

Not that these responses, aside from the last one relied on this law being invoked so quickly. Even if LLMs ‘on their own’ do proved unable to ever solve such problems, which would have been super weird? So what? They could still serve as the core engine that then introduces scaffolding and tools to allow them to get such abilities and solve such problems, and generally deal with unexpected new logic-style problems, and other types of new problems as well.

Or: If, as many say, current AI is bad at what sci-fi computers are good at, and good at what those computers are bad at, you can fix this by hooking them up to a computer.

Victor then explained that no, the point was not to massage an LLM into solving that one particular instance of the A::B prompting challenge. The point was to be able to reliably and systematically solve such problems in general.

Then things got more interesting. This was not all talk. Let’s go.

Victor Taelin: A::B Prompting Challenge: $10k to prove me wrong!

# CHALLENGE

Develop an AI prompt that solves random 12-token instances of the A::B problem (defined in the quoted tweet), with 90%+ success rate.

# RULES

1. The AI will be given a random instance, inside a tag.

2. The AI must end its answer with the correct .

3. The AI can use up to 32K tokens to work on the problem.

4. You can choose any public model.

5. Any prompting technique is allowed.

6. Keep it fun! No toxicity, spam or harassment.

# EVALUATION

You must submit your system prompt as a reply to this tweet, in a Gist. I’ll test each submission in 50 random 12-token instances of the A::B system. The first to get 45 correct solutions wins the prize, plus the invaluable public recognition of proving me wrong 😅 If nobody solves it, I’ll repost the top 3 submissions, so we all learn some new prompting techniques 🙂

And then, about a day later, he did made good, paying out and admitting he was wrong.

Victor Taelin: I *WASWRONG – $10K CLAIMED!

## The Claim Two days ago, I confidently claimed that “GPTs will NEVER solve the A::B problem”. I believed that: 1. GPTs can’t truly learn new problems, outside of their training set, 2. GPTs can’t perform long-term reasoning, no matter how simple it is. I argued both of these are necessary to invent new science; after all, some math problems take years to solve. If you can’t beat a 15yo in any given intellectual task, you’re not going to prove the Riemann Hypothesis. To isolate these issues and raise my point, I designed the A::B problem, and posted it here – full definition in the quoted tweet.

## Reception, Clarification and Challenge

Shortly after posting it, some users provided a solution to a specific 7-token example I listed. I quickly pointed that this wasn’t what I meant; that this example was merely illustrative, and that answering one instance isn’t the same as solving a problem (and can be easily cheated by prompt manipulation).

So, to make my statement clear, and to put my money where my mouth is, I offered a $10k prize to whoever could design a prompt that solved the A::B problem for *random12-token instances, with 90%+ success rate. That’s still an easy task, that takes an average of 6 swaps to solve; literally simpler than 3rd grade arithmetic. Yet, I firmly believed no GPT would be able to learn and solve it on-prompt, even for these small instances.

## Solutions and Winner

Hours later, many solutions were submitted. Initially, all failed, barely reaching 10% success rates. I was getting fairly confident, until, later that day, @ptrschmdtnlsn and @SardonicSydney submitted a solution that humbled me. Under their prompt, Claude-3 Opus was able to generalize from a few examples to arbitrary random instances, AND stick to the rules, carrying long computations with almost zero errors. On my run, it achieved a 56% success rate.

Through the day, users @dontoverfit (Opus), @hubertyuan_ (GPT-4), @JeremyKritz (Opus) and @parth007_96 (Opus), @ptrschmdtnlsn (Opus) reached similar success rates, and @reissbaker made a pretty successful GPT-3.5 fine-tune. But it was only late that night that @futuristfrog posted a tweet claiming to have achieved near 100% success rate, by prompting alone. And he was right. On my first run, it scored 47/50, granting him the prize, and completing the challenge.

## How it works!?

The secret to his prompt is… going to remain a secret! That’s because he kindly agreed to give 25% of the prize to the most efficient solution. This prompt costs $1+ per inference, so, if you think you can improve on that, you have until next Wednesday to submit your solution in the link below, and compete for the remaining $2.5k! Thanks, Bob.

## How do I stand?

Corrected! My initial claim was absolutely WRONG – for which I apologize. I doubted the GPT architecture would be able to solve certain problems which it, with no margin for doubt, solved. Does that prove GPTs will cure Cancer? No. But it does prove me wrong! Note there is still a small problem with this: it isn’t clear whether Opus is based on the original GPT architecture or not. All GPT-4 versions failed. If Opus turns out to be a new architecture… well, this whole thing would have, ironically, just proven my whole point 😅.

But, for the sake of the competition, and in all fairness, Opus WAS listed as an option, so, the prize is warranted.

## Who I am and what I’m trying to sell? Wrong! I won’t turn this into an ad. But, yes, if you’re new here, I AM building some stuff, and, yes, just like today, I constantly validate my claims to make sure I can deliver on my promises. But that’s all I’m gonna say, so, if you’re curious, you’ll have to find out for yourself (:

#### That’s all. Thanks for all who participated, and, again – sorry for being a wrong guy on the internet today! See you. Gist:

Excellent all around. Again, this is The Way.

I wish more of the claims that mattered were this tangible and easy to put to the test. Alas, in many cases, there is no similar objective test. Nor do I expect most people who proudly assert things similar to Victor’s motivating claim to update much on this, even if it comes to their attention. Still, we do what we can.

Two things this shows is how good and quick a motivated internet is at unlocking the latent capabilities of models, and that those latent capabilities are often much better than we might think. If you give them motivation, a lot of people will suddenly get very creative, smart and dedicated. Think about the time frames here. A few hours in, Victor was getting very confident. A day later, it was over.

This was also a test of various models. What would people use when there were real stakes and they needed to solve a real problem? Most people who got anywhere chose Claude Opus, although we do have one solid attempt with GPT-4 and one fine-tune of GPT-3.5. It seems increasingly clear, from many angles, that Claude Opus is currently our best option when we don’t care about marginal inference costs.

Aurora-M is claimed to be ‘red teamed in accordance with the Executive Order.’ As Jack Clark discovers, this is actually Anthropic’s red team data set in a trenchcoat, developed before the Executive Order, not even tailored to ‘address concerns’ from the Executive Order.

We will increasingly need to watch out for this kind of glaring falsification of the spirit when looking at safety efforts. There is nothing wrong with using Anthropic’s red teaming data set on Aurora-M, but when you start this kind of labeling trouble follows.

I do not understand what Davidad is trying to advocate for here in terms of using practical politics to ensure we take our technological gains in the form of safety, and share Emmett Shear’s confusions and also others, but passing it along.

I have previously gotten pushback about putting Richard Sutton in this section.

No, people say. You have it wrong. Richard Sutton does not argue for or favor human extinction. He simply predicts it and thinks we should accept that it will happen.

Or, alternatively, he is not arguing in favor of human extinction. He is only arguing in favor of a policy regime he believes inevitably would lead to rapid human extinction, and he thinks we should ‘prepare for’ that outcome rather than attempt to prevent it.

To which my response is, okay, fine, I guess. Let’s go to the videotape and judge?

Existential Risk Observatory: It’s high time that people like Larry Page, Hans Moravec, @RichardSSutton, and @SchmidhuberAI are called out, not to mention @BasedBeffJezos and e/acc. These are not respectable scientists and industrialists. They are arguing for human extinction, which should never be acceptable. In many cases, they are even actively helping to bring about human extinction by working on species-threatening AGI without doing enough to keep it under our control, which should never be acceptable, either.

Richard Sutton (April 7): Nobody is arguing in favor of human extinction. The disagreement is between those who want centralized control of AI, like yourself, and those who want decentralization, in particular, those who want permissionless innovation.*

Yanco: You’re a liar, sir.

[Quotes Richard Sutton from this video from eight years ago]: “[AIs] might tolerate us as pets or workers. (…) If we are useless, and we have no value [to the AI] and we’re in the way, then we would go extinct, but maybe that’s rightly so.”

Yanco: A man that is perfectly fine w/ AIs murdering you & your children.

Richard Sutton (Tweet from September 8, 2023): We should prepare for, but not fear, the inevitable succession from humanity to AI, or so I argue in this talk pre-recorded for presentation at WAIC in Shanghai. [links to this YouTube video, called AI Succession]

Connor Leahy (April 9): Do these liars not think we keep receipts?

Are Yanco and Connor being slightly unnuanced and uncharitable here? Yes.

In contrast, what Richard Sutton is doing here is best called ‘gaslighting’ and ‘lying.’

It would be fair to say that Sutton does not seem enthused about the prospect, he is not exactly ‘perfectly fine’ with the murder spree. I am confident he is a decent enough person that if his choice was to have or not have a universal murder spree, he would choose no murder spree.

He simply wants ‘permissionless innovation’ rather than ‘centralized control.’ Except that he himself knows, and says out loud, that his ‘permissionless innovation’ would cause human extinction. And he says, ‘we should not resist this succession.’

There are others who genuinely think that AI does not pose a risk of human extinction. In which case, I disagree strongly, but that is a fact disagreement. That does not apply to Richard Sutton.

If you go to the linked article (HT: Richard Sutton, basically) you see this described in very careful words. Here is its part referring to Sutton.

Emile Torres: Other computer scientists have promoted the same view. Richard Sutton, who is highly respected within a subfield of AI called “reinforcement learning,” argues that the “succession to AI is inevitable.” Though these machines may “displace us from existence,” he tells us that “we should not resist [this] succession.” Rather, people should see the inevitable transformation to a new world run by AIs as “beyond humanity, beyond life, beyond good and bad.” Don’t fight against it, because it cannot be stopped.

That seems like a fair and precise description. The response was gaslighting.

The rest of the piece is good as well, although I have not observed much of a key phenomenon he warns about, which is the idea that people will claim ‘humanity’ and ‘human extinction’ should count any future digital beings as humans, often as a hidden implicit assumption.

I am willing to simply say that if you use words like that, you are lying, that is not what ‘human’ means, and if you think such an outcome is fine you should be willing to call it by its true name. Then we can discuss whether the successor you are proposing is something to which we ascribe value, and exactly what outcomes we have what value to us, and choose what to do based on that.

I’m dying out here. Also that’s not the least worrisome rule of three.

Roon: If you’re not playing you’re dying.

If you’re not producing art you’re dying.

If you don’t love power like a violinist loves his violin you’re dying.

Another view: If you love power like a violinist loves her violin, you are already dead.

Leaving this here for future reference.

All right, fine, yes, this one is a banger.

Kache: Suno ai music generation is going to revolutionize bullying.

“ok computer, make a song making fun of johnny two shoes. we call him two shoes because he wore mismatching shoes one day. also his mom died last week”

Help wanted.

Eliezer Yudkowsky: Yes yes, there’s other hypotheses for how this could happen; but I still wonder if part of the problem is that people who are just hearing about AI believe:

– computers were always sort of like this

– ChatGPT is just doing more of it

– this all happened much slower than it did.

Spencer Schiff: Yes and they have no conception of the rate of improvement.

AI #59: Model Updates Read More »

NASA knows what knocked Voyager 1 offline, but it will take a while to fix

jet propulsion laboratory, memory, NASA, Science, Space, Voyager, Voyager 1 / Paul Patrick / April 5, 2024

Hope returns —

“Engineers are optimistic they can find a way for the FDS to operate normally.”

Stephen Clark – Apr 6, 2024 12: 28 am UTC

Enlarge / A Voyager space probe in a clean room at the Jet Propulsion Laboratory in 1977.

Engineers have determined why NASA’s Voyager 1 probe has been transmitting gibberish for nearly five months, raising hopes of recovering humanity’s most distant spacecraft.

Voyager 1, traveling outbound some 15 billion miles (24 billion km) from Earth, started beaming unreadable data down to ground controllers on November 14. For nearly four months, NASA knew Voyager 1 was still alive—it continued to broadcast a steady signal—but could not decipher anything it was saying.

Confirming their hypothesis, engineers at NASA’s Jet Propulsion Laboratory (JPL) in California confirmed a small portion of corrupted memory caused the problem. The faulty memory bank is located in Voyager 1’s Flight Data System (FDS), one of three computers on the spacecraft. The FDS operates alongside a command-and-control central computer and another device overseeing attitude control and pointing.

The FDS duties include packaging Voyager 1’s science and engineering data for relay to Earth through the craft’s Telemetry Modulation Unit and radio transmitter. According to NASA, about 3 percent of the FDS memory has been corrupted, preventing the computer from carrying out normal operations.

Optimism growing

Suzanne Dodd, NASA’s project manager for the twin Voyager probes, told Ars in February that this was one of the most serious problems the mission has ever faced. That is saying something because Voyager 1 and 2 are NASA’s longest-lived spacecraft. They launched 16 days apart in 1977, and after flying by Jupiter and Saturn, Voyager 1 is flying farther from Earth than any spacecraft in history. Voyager 2 is trailing Voyager 1 by about 2.5 billion miles, although the probes are heading out of the Solar System in different directions.

Normally, engineers would try to diagnose a spacecraft malfunction by analyzing data it sent back to Earth. They couldn’t do that in this case because Voyager 1 has been transmitting data packages manifesting a repeating pattern of ones and zeros. Still, Voyager 1’s ground team identified the FDS as the likely source of the problem.

The Flight Data Subsystem was an innovation in computing when it was developed five decades ago. It was the first computer on a spacecraft to use volatile memory. Most of NASA’s missions operate with redundancy, so each Voyager spacecraft launched with two FDS computers. But the backup FDS on Voyager 1 failed in 1982.

Due to the Voyagers’ age, engineers had to reference paper documents, memos, and blueprints to help understand the spacecraft’s design details. After months of brainstorming and planning, teams at JPL uplinked a command in early March to prompt the spacecraft to send back a readout of the FDS memory.

The command worked, and Voyager.1 responded with a signal different from the code the spacecraft had been transmitting since November. After several weeks of meticulous examination of the new code, engineers pinpointed the locations of the bad memory.

“The team suspects that a single chip responsible for storing part of the affected portion of the FDS memory isn’t working,” NASA said in an update posted Thursday. “Engineers can’t determine with certainty what caused the issue. Two possibilities are that the chip could have been hit by an energetic particle from space or that it simply may have worn out after 46 years.”

Voyager 1’s distance from Earth complicates the troubleshooting effort. The one-way travel time for a radio signal to reach Voyager 1 from Earth is about 22.5 hours, meaning it takes roughly 45 hours for engineers on the ground to learn how the spacecraft responded to their commands.

NASA also must use its largest communications antennas to contact Voyager 1. These 230-foot-diameter (70-meter) antennas are in high demand by many other NASA spacecraft, so the Voyager team has to compete with other missions to secure time for troubleshooting. This means it will take time to get Voyager 1 back to normal operations.

“Although it may take weeks or months, engineers are optimistic they can find a way for the FDS to operate normally without the unusable memory hardware, which would enable Voyager 1 to begin returning science and engineering data again,” NASA said.

NASA knows what knocked Voyager 1 offline, but it will take a while to fix Read More »

$158,000 ALS drug pulled from market after failing in large clinical trial

ALS, Amylyx, clinical trial, fda, health, Science / Paul Patrick / April 5, 2024

Off the market —

The drug is now unavailable to new patients; its maker to lay off 70% of employees.

Beth Mole – Apr 5, 2024 10: 28 pm UTC

Amylyx, the maker of a new drug to treat ALS, is pulling that drug from the market and laying off 70 percent of its workers after a large clinical trial found that the drug did not help patients, according to an announcement from the company Thursday.

The drug, Relyvrio, won approval from the Food and Drug Administration in September 2022 to slow the progression of ALS (amyotrophic lateral sclerosis, or Lou Gehrig’s disease). However, the data behind the controversial decision was shaky at best; it was based on a study of just 137 patients that had several weaknesses and questionable statistical significance, and FDA advisors initially voted against approval. Still, given the severity of the neurogenerative disease and lack of effective treatments, the FDA ultimately granted approval under the condition that the company was working on a Phase III clinical trial to solidify its claimed benefits.

Relyvrio—a combination of two existing, generic drugs—went on the market with a list price of $158,000.

Last month, the company announced the top-line results from that 48-week, randomized, placebo-controlled trial involving 664 patients: Relyvrio failed to meet any of the trial’s goals. The drug did not improve patients’ physical functions, which were scored on a standardized ALS-specific test, nor did it improve quality of life, respiratory function, or overall survival. At that time, the co-CEOs of the company said they were “surprised and deeply disappointed” by the result, and the company acknowledged that it was considering voluntarily withdrawing the drug from the market.

In the announcement on Thursday, the company called Relyvrio’s market withdrawal a “difficult moment for the ALS community.” Patients already taking the medication who wish to continue taking it will be able to do so through a free drug program, the company said. It is no longer available to new patients, effective Thursday.

Amylyx is now “restructuring” to focus on two other drug candidates that treat different neurodegenerative disease. The change will include laying off 70 percent of its workforce, which, according to The Washington Post, includes more than 350 employees.

Relyvrio is part of a series of similarly controversial drugs for devastating neurodegenerative diseases that have gained FDA approval despite questionable data. In January, drug maker Biogen announced it was abandoning Aduhelm, a highly contentious Alzheimer’s drug that failed two large trials prior to its heavily criticized approval.

$158,000 ALS drug pulled from market after failing in large clinical trial Read More »

Apple now allows retro game emulators on its App Store—but with big caveats

App Store, Apple, Apple App Store, Emulation, gaming, iOS, ipad, iPadOS, IPhone, retro games, Tech / Paul Patrick / April 5, 2024

RETRO GAMES —

It’s probably not the Wild West of game emulation you’re hoping for. Here’s why.

Samuel Axon – Apr 5, 2024 10: 14 pm UTC

A screenshot of Sonic the Hedgehog on an iPhone — Enlarge / The classic Sega Genesis game *Sonic the Hedgehog* running on an iPhone—in this case, as a standalone app.

Samuel Axon

When Apple posted its latest update to the App Store’s app review and submission policies for developers, it included language that appears to explicitly allow a new kind of app for emulating retro console games.

Apple has long forbidden apps that run code from an external source, but today’s announced changes now allow “software that is not embedded in the binary” in certain cases, with “retro game console emulator apps can offer to download games” specifically listed as one of those cases.

Here’s the exact wording:

4.7 Mini apps, mini games, streaming games, chatbots, plug-ins, and game emulators

Apps may offer certain software that is not embedded in the binary, specifically HTML5 mini apps and mini games, streaming games, chatbots, and plug-ins. Additionally, retro game console emulator apps can offer to download games. You are responsible for all such software offered in your app, including ensuring that such software complies with these Guidelines and all applicable laws. Software that does not comply with one or more guidelines will lead to the rejection of your app. You must also ensure that the software adheres to the additional rules that follow in 4.7.1 and 4.7.5. These additional rules are important to preserve the experience that App Store customers expect, and to help ensure user safety.

It’s a little fuzzy how this will play out, but it may not allow the kind of emulators you see on Android and desktop, which let you play retro games from any outside source.

Retro game emulators run what are colloquially called ROM files, which are more or less images of the game cartridges or discs that played on console hardware. By now, it’s well-established that the emulators themselves are completely legal, but the legality of the ROM files downloaded from ROM sites on the Internet depends on the specific files and circumstances.

There are ROMs that are entirely public domain or in some license where the creator allows distribution; there are ROMs that are technically copyrighted intellectual property but where the original owner no longer exists, and the current ownership is unknown or unenforced; and there are some ROMs (like many games made by Nintendo) where the owner still has an interest in controlling distribution and often takes action to try to curb illegal sharing and use of the files.

Additionally, many game publishers use emulators to run ROMs of their own retro games, which they sell to consumers either as standalone games or in collections for modern platforms.

It’s not completely clear from Apple’s wording, but our interpretation of Apple’s new rules is that it’s likely only the last of those examples will be possible; companies that own the intellectual property could launch emulator apps for downloading ROMs of their (and only their) games. So, for example, Sega could offer a Sega app that would allow users to download an ever-expanding library of Sega games, either as part of a subscription, for free, or as in-app purchases. Sega has offered its retro games on the iPhone before in emulation but with a standalone app for each game.

“You are responsible for all such software offered in your app, including ensuring that such software complies with these Guidelines and all applicable laws,” Apple writes. And it specifically says “retro game console emulator apps can offer to download games” in the list of exceptions to the rules against “software that is not embedded inside the binary”—but it doesn’t list any other method for retro game console emulator apps.

Whatever the case, this update is not limited to the European Union. Apple has been subjected to regulatory scrutiny in both the EU and the United States regarding its App Store rules. It’s likely the company is making this change to preempt criticism in this area, though it did not name its reasons when announcing the change other than to say it has been made to “support updated policies, upcoming features, and to provide clarification.”

Apple now allows retro game emulators on its App Store—but with big caveats Read More »

Roku has patented a way to show ads over anything you plug into your TV

ads, Roku, roku tv, Tech / Paul Patrick / April 5, 2024

giving us pause —

System would detect paused content on external devices and show ads on top.

Andrew Cunningham – Apr 5, 2024 9: 47 pm UTC

Enlarge / A promotional image for a Roku TV.

Roku TV sets come with ads. Generally, these are restricted to Roku’s home and menu screens, its screensavers, and its first-party video channels, and once you start playing video, the only ads you’ll see are the ones from the service you’re streaming from. That said, Roku TVs have shown ads atop live TV before.

Now, the company is apparently experimenting with ways to show ads over top of even more of the things you plug into your TV. A patent application from the company spotted by Lowpass describes a system for displaying ads over any device connected over HDMI, a list that could include cable boxes, game consoles, DVD or Blu-ray players, PCs, or even other video streaming devices. Roku filed for the patent in August 2023 and it was published in November 2023, though it hasn’t yet been granted.

The technology described would detect whether content was paused in multiple ways—if the video being displayed is static, if there’s no audio being played, if a pause symbol is shown anywhere on screen, or if (on a TV with HDMI-CEC enabled) a pause signal has been received from some passthrough remote control. The system would analyze the paused image and use metadata “to identify one or more objects” in the video frame, transmit that identification information to a network, and receive and display a “relevant ad” over top of whatever the paused content is.

Enlarge / The proposed Roku device would include multiple modules dedicated to detecting and analyzing onscreen content and inserting ads over top of an existing video stream.

Roku

This theoretical Roku TV’s internal hardware would be capable of taking the original source video feed, rendering an ad, and then combining the two into a single displayed image. Combining those video streams could enable both static or animated ads, according to the patent.

Patents are just patents. A filed and granted patent isn’t the same thing as a concrete plan to actually implement the technology described in the patent. We could see this feature come to future Roku TV sets exactly as described, or we could never hear about it again. Everything from the cost of implementing the feature, to difficulty making it work in real life the way it works on paper, to user and partner backlash could dissuade Roku from putting this into practice.

But Roku the company is highly motivated to figure out more ways to make more money from more ads on Roku devices. Among the business risks disclosed on Roku’s financial filings from its 2023 fiscal year (PDF), the company says that its “future growth depends on the acceptance and growth of streaming TV advertising and advertising platforms.”

Roku's proposed workflow for detecting paused content and displaying an ad on top of it. — Enlarge / Roku’s proposed workflow for detecting paused content and displaying an ad on top of it.

Roku

If implemented as described, this system both gives Roku another place to put ads, and gives the company another source of user data that can be used to encourage advertisers to spend on its platforms. Roku also anticipates that the general shift toward ad-supported streaming tiers that we’ve seen in the last couple of years will “shift… ad dollars from traditional TV to streaming,” (PDF) and having more places to put ads will put Roku in a better position to capitalize on that shift.

It’s unclear whether this kind of feature could be enabled on currently supported Roku TVs with a software update, or if it would require a newer set with newer hardware in it. It seems as though a Roku TV that was capable of this kind of ad insertion would need more sophisticated internal hardware than most current sets currently come with—this is the same company that feuded with Google a few years back because it didn’t want to pay for more-expensive chips that could decode Google’s AV1 video codec.

Roku has patented a way to show ads over anything you plug into your TV Read More »

Will there be a Dune: Part Three? Yes… with caveats on timing

culture, denis villeneuve, Dune, Dune: Messiah, Entertainment, Frank Herbert, Legendary Pictures / Paul Patrick / April 5, 2024

Prepare for the Messiah —

Timing of a big-screen return to Arrakis mostly depends on director Denis Villeneuve.

Jennifer Ouellette – Apr 5, 2024 9: 26 pm UTC

Legendary Pictures has confirmed that it plans to make <em>Dune: Part Three</em> with director Denis Villeneuve.” src=”https://cdn.arstechnica.net/wp-content/uploads/2023/12/dune2-800×537.jpg”></img><figcaption>
<p><a data-height=

Dune: Part Two is still raking in the moolah at the box office, and deservedly so. But judging by my various feeds, fans are already swooning over the prospect of director Denis Villeneuve extending his vision into a trilogy by adapting Frank Herbert’s 1969 sequel, Dune Messiah, for the next installment. Will there be a Dune: Part Three? Most signs currently point to yes, with a couple of caveats. Exactly how soon we’ll be seeing a return to Arrakis depends a lot on Villeneuve.

Variety confirmed that Legendary Pictures is working with the director on developing Dune: Part Three, although it remains unclear from the wording of the plethora of news items whether the project has officially been greenlit. (“Development” can mean a lot of things.) Naturally, the studio is eager, as are we: the film is the biggest hit of 2024 thus far, with global earnings of $630 million (although the hotly anticipated Deadpool and Wolverine this summer might give it a run for its money).

That confirmation sent fresh frissons of excitement across the Internet, although Villeneuve had been talking about the prospect as far back as September 2021. Those plans always depended on the success of Part Two, and that hurdle has obviously been cleared. By August 2023, the director was on record saying there were “words on paper” for a third film. And we learned just last month that composer Hans Zimmer was already working on the score for Dune: Part Three.

That said, Villeneuve has yet to commit to an actual release date publicly, emphasizing his desire to take a little break from the Dune franchise to work on something else. (Per Variety, Legendary has already tapped him to adapt Anne Jacobsen’s nonfiction book, Nuclear War: A Scenario, but that project will likely come after Dune: Part Three.) He even hinted at one point that it might be a good idea to let star Timothée Chalamet age a bit, given the 12-year gap in the novels.

And in February, Villeneuve told The Times UK that while he had nearly finished a draft script (“barely an embryo”), he was not inclined to rush things: “I want to make sure that if we go back there a third time that it’ll be worth it, and that it would be make something even better than Part Two.” That’s a tall order, given the critical raves that have accompanied the film’s box office success. But we’re betting Villeneuve can pull it off… in his own good time.

Will there be a Dune: Part Three? Yes… with caveats on timing Read More »

Tesla scraps its plan for a $25,000 Model 2 EV

Cars, Tesla / Paul Patrick / April 5, 2024

who believes a word Elon Musk says? —

Musk says that “Reuters is lying (again)” over reports that the Model 2 is dead.

Jonathan M. Gitlin – Apr 5, 2024 7: 12 pm UTC

Budrul Chukrut/SOPA Images/LightRocket via Getty Images

Tesla has abandoned plans to develop an affordable electric Model 2, according to a report in Reuters. The news organization says it has reviewed company messages that say the affordable Model Y, which Tesla CEO Elon Musk claimed would sell for $25,000 or less, has been axed.

Musk has been talking about an affordable Tesla Model 2 for some time now. An affordable mass-market EV was supposedly always key to the company’s long-term “master plans,” and in December 2023, he said the company was working on a “low-cost electric vehicle that will be made at very high volume.” Then, this March, Musk told Tesla workers that the Model 2 would go into production at the company’s factory in Berlin.

In light of this news, that statement certainly raises eyebrows—Reuters reports that one of its three unnamed sources told it that the decision to scrap the Model 2 was made in late February. Instead, Musk is allegedly “all in on robotaxi,” Tesla’s plan to create an autonomous driving system that could allow its cars to compete with Uber or Lyft without a driver in the equation.

Tesla has no press office and has not rebutted the news, but Musk took to his social network to declare that “Reuters is dying,” then in another post claimed that “Reuters is lying (again).”

Earlier this week, Tesla posted its worst delivery results since 2020, with an 8.5 percent drop in deliveries year over year and yet another quarter of overproduction that has left the electric carmaker with nearly 150,000 vehicles produced but unsold.

The next few weeks may offer little respite for Musk or Tesla; the trial over the death of Apple engineer Walter Huang, who was killed when his Tesla Model X drove into a highway gore in 2018, gets underway in California on Monday.

Tesla scraps its plan for a $25,000 Model 2 EV Read More »