Blake Jones

These spiders listen for prey before hurling webs like slingshots

Science / Blake Jones / December 4, 2024

Along came a spider

A) Untensed web shown from front view. (B) Tensed web shown from side view. Credit: S.I. Han and T.A. Blackledge, 2024

The 19 spiders built 26 webs over the testing period. For the experiments, Han and Blackledge used a weighted tuning fork with frequencies in the mid-range for whirring wings for many mosquito species in North America as a control stimulus. They also attached actual mosquitos to thin strips of black construction paper by dabbing a bit of superglue on their abdomens or hind legs. This ensured the mosquitos could still beat their wings when approaching the webs. The experiments were recorded on high-speed video for analysis.

As expected, spiders released their webs when flapping mosquitoes drew near, but the video footage showed that the releases occurred before the mosquitoes ever touched the web. The spiders released their webs just as frequently when the tuning fork was brandished nearby. It wasn’t likely that they were relying on visual cues because the spiders were centered at the vertex of the web and anchor line, facing away from the cone. Ray spiders also don’t have well-developed eyes. And one spider did not respond to a motionless mosquito held within the capture cone but released its web only when the insect started flapping its wings.

“The decision to release a web is therefore likely based upon vibrational information,” the authors concluded, noting that ray spiders have sound-sensitive hairs on their back legs that could be detecting air currents or sound waves since those legs are typically closest to the cone. Static webs are known to vibrate in response to airborne sounds, so it seems likely that ray spiders can figure out an insect’s approach, its size, or maybe even its behavior before the prey ever makes contact with the web.

As for the web kinematics, Han and Blackledge determined that they can accelerate up to 504 m/s², reaching speeds as high as 1 m/s, and hence can catch mosquitos in 38 milliseconds or less. Even the speediest mosquitoes might struggle to outrun that.

Journal of Experimental Biology, 2024. DOI: 10.1242/jeb.249237 (About DOIs).

These spiders listen for prey before hurling webs like slingshots Read More »

OpenAI teases 12 days of mystery product launches starting tomorrow

AI, Biz & IT, chatgpt, chatgtp, Christmas, machine learning, openai / Blake Jones / December 4, 2024

On Wednesday, OpenAI CEO Sam Altman announced a “12 days of OpenAI” period starting December 5, which will unveil new AI features and products for 12 consecutive weekdays.

Altman did not specify the exact features or products OpenAI plans to unveil, but a report from The Verge about this “12 days of shipmas” event suggests the products may include a public release of the company’s text-to-video model Sora and a new “reasoning” AI model similar to o1-preview. Perhaps we may even see DALL-E 4 or a new image generator based on GPT-4o’s multimodal capabilities.

Altman’s full tweet included hints at releases both big and small:

🎄🎅starting tomorrow at 10 am pacific, we are doing 12 days of openai.

each weekday, we will have a livestream with a launch or demo, some big ones and some stocking stuffers.

we’ve got some great stuff to share, hope you enjoy! merry christmas.

If we’re reading the calendar correctly, 12 weekdays means a new announcement every day until December 20.

OpenAI teases 12 days of mystery product launches starting tomorrow Read More »

Amazon secretly slowed deliveries, deceived anyone who complained, lawsuit says

Amazon, Amazon Prime, Policy, unfair business practices / Blake Jones / December 4, 2024

In a statement to Ars, Amazon spokesperson Kelly Nantel said that claims that Amazon’s “business practices are somehow discriminatory or deceptive” are “categorically false.”

Nantel said that Amazon started using third-party services to deliver to these areas to “put the safety of delivery drivers first.”

“In the ZIP codes in question, there have been specific and targeted acts against drivers delivering Amazon packages,” Nantel said. “We made the deliberate choice to adjust our operations, including delivery routes and times, for the sole reason of protecting the safety of drivers.”

Nantel also pushed back on claims that Amazon concealed this choice, claiming that the company is “always transparent with customers during the shopping journey and checkout process about when, exactly, they can expect their orders to arrive.”

But that doesn’t really gel with Schwalb’s finding that even customers using Amazon’s support chat were allegedly misled. During one chat, a frustrated user pointing out discrepancies between DC ZIP codes asked if Amazon “is a waste of money in my zip code?” Instead of confirming that the ZIP code was excluded from in-house delivery services, the support team member seemingly unhelpfully suggested the user delete and re-add their address to their account.

“Amazon has doubled down on its deception by refusing to disclose the fact of the delivery exclusion, and instead has deceptively implied that slower speeds are simply due to other circumstances, rather than an affirmative decision by Amazon,” Schwalb’s complaint said.

Schwalb takes no issue with Amazon diverting delivery drivers from perceived high-crime areas but insists that Amazon owes its subscribers in those regions an explanation for delivery delays and perhaps even cheaper subscription prices. He has asked for an injunction on Amazon’s allegedly deceptive advertising urging users to pay for fast shipments they rarely, if ever, receive. He also wants Amazon to refund subscribers seemingly cheated out of full subscription benefits and has asked a jury to award civil damages to deter future unfair business practices. Amazon could owe millions in a loss, with each delivery to almost 50,000 users since mid-2022 considered a potential violation.

Nantel said that Amazon has offered to “work together” with Schwalb’s office “to reduce crime and improve safety in these areas” but did not suggest Amazon would be changing how it advertises Prime delivery in the US. Instead, the e-commerce giant plans to fight the claims and prove that “providing fast and accurate delivery times and prioritizing the safety of customers and delivery partners are not mutually exclusive,” Nantel said.

Amazon secretly slowed deliveries, deceived anyone who complained, lawsuit says Read More »

Seagrass is fantastic at carbon capture—and it’s at risk of extinction

climate change, eel grass, Science, sea grass, syndication / Blake Jones / December 4, 2024

An underwater gardening experiment along the East Coast aims at restoration.

A crab inhabits a bed of eelgrass at Cape Cod National Seashore in Massachusetts. Eelgrass provides critical habitat for hundreds of species. Credit: Holly Plaisted/National Park Service

In late September, seagrass ecologist Alyssa Novak pulled on her neoprene wetsuit, pressed her snorkel mask against her face, and jumped off an oyster farming boat into the shallow waters of Pleasant Bay, an estuary in the Cape Cod National Seashore in Massachusetts. Through her mask she gazed toward the sandy seabed, about 3 feet below the surface at low tide, where she was about to plant an experimental underwater garden of eelgrass.

Naturally occurring meadows of eelgrass—the most common type of seagrass found along the East Coast of the United States—are vanishing. Like seagrasses around the world, they have been plagued for decades by dredging, disease, and nutrient pollution from wastewater and agricultural runoff. The nutrient overloads have fueled algal blooms and clouded coastal waters with sediments, blocking out sunlight the marine plants need to make food through photosynthesis and suffocating them.

The United Nations Environment Program reports more than 20 of the world’s 72 seagrass species are on the decline. As a result, an estimated 7 percent of these habitats are lost each year.

In the western Atlantic, some eelgrass meadows have been reduced by more than 90 percent in the last 100 years, according to The Nature Conservancy, an environmental nonprofit that works to protect lands and waters around the world.

Now, rising sea surface temperatures caused by global warming are pushing the plant to the brink of extinction. Novak, a research assistant professor at Boston University who has studied eelgrass in New England for more than a decade, and a multidisciplinary team of scientists in different states are trying their best to make sure this does not become reality.

Together, they are working to restore eelgrass populations in coastal parks from Maine to North Carolina using a novel approach that has never been tried before with a marine plant: assisted migration.

“We’re trying to identify thermo-tolerant individuals up and down the East Coast and try to move them into areas where the populations are stressed by increases in sea surface temperature, so that we can give those populations a chance of surviving into the future,” Novak said.

Typically, eelgrass thrives in water temperatures between 60° and 68° Fahrenheit, according to Novak. In the last 20 years, sea surface temperatures in the Northeast have warmed faster than the global ocean and exceeded that safe range, mostly due to human activity like burning fossil fuels, according to NOAA Fisheries, a federal agency charged with managing and protecting marine resources in the US.

Blades of eelgrass are viewed up close at Cape Cod National Seashore. Credit: Holly Plaisted/National Park Service

Around 77° Fahrenheit the plants become stressed and struggle to photosynthesize, said Novak. Around 82° they begin to expire. “That’s when the plants no longer can handle the heat stress, and they end up dying,” she said. And it’s getting hotter.

In recent years, she said, water temperatures along the East Coast have surpassed 82° during peak summer months. By 2050, they are expected to increase in the Northeast by two degrees, she said.

The common garden experiment

Anticipating the deadly forecast for eelgrass, The Nature Conservancy brought together a group of scientists in 2022 to figure out how they might change the plant’s trajectory. Together, the experts on seagrasses, corals, agriculture, forestry, and plant genetics explored options based on what had been done to address the effects of climate change on other ecosystems.

“We wanted to figure out what the solutions were that different groups had come up with, and from those, which ones might apply to the seagrass world,” said Boze Hancock, senior marine restoration scientist with The Nature Conservancy’s global oceans team.

Prolonged marine heatwaves and coral disease have prompted some scientists to experiment with cross-breeding and replanting heat-resistant corals in warming waters, for example. In some cases they have removed whole coral colonies from their natural habitat to preserve their genetics in land-based biobanks.

One of the workshop invitees, biologist Thomas Whitham, shared with the group how he’s used a scientific research tool called the “common garden experiment” to restore deciduous Fremont cottonwood forests that have been dying off in Arizona due to rising temperatures and drought.

The experiments involve collecting plants from different locations and moving them to designated locations to observe how they respond to new environmental conditions. In the case of Fremont cottonwoods, Whitham said the technique has proven vital to identifying trees with specific genetic traits that make them more heat and drought resilient. Cuttings from these trees are now being planted in areas where less resilient trees died off to restore the species in a process known as “assisted migration.”

“We’ve planted many thousands, tens of thousands, of trees using this common garden approach,” said Whitham, a Regents’ professor in the department of biological sciences at Northern Arizona University. It could work for eelgrass too, he told the group.

They could collect seeds from eelgrass populations in the south and plant them in cooler northern waters alongside local seeds and, in effect, identify plants that have a propensity to thrive in warmer temperatures.

Workshop participants were eager to try, said attendee Jonathan Lefcheck, a research scientist at the University of Maryland Center for Environmental Science who has studied seagrasses in the Chesapeake Bay for more than 15 years. “If we do nothing, it’s likely that seagrass—eelgrass—will be extirpated all the way up to New York in the next 50 years,” he said. And with it, all the services it provides to wildlife and humans.

Underwater forests

Eelgrass provides critical habitat for hundreds of species.

“It’s the forest under the water in the estuaries,” said Bradley Peterson, a professor of marine science at Stony Brook University’s School of Marine and Atmospheric Sciences who helped initiate the workshops in collaboration with The Nature Conservancy.

Scientists believe seagrasses evolved from terrestrial plants 70 to 100 millions years ago. “When they went into the marine world, they brought all the machinery they had with them for the terrestrial world, real seeds, real flowers, and real roots,” said Peterson, who is working to restore eelgrass near Long Island.

Its green grass blades, which can grow up to a couple feet long, offer food and shelter to horseshoe crabs, seahorses, and fish of all sizes that weave through its mazes. Little shrimp pollinate the plant’s flowers like “bees of the sea,” said Lefcheck. For bigger fish, “it’s this beautiful buffet,” he said. “You get this whole ecosystem that’s built up around this habitat that’s just sort of gently swaying there underneath the waves.”

In New England, eelgrass is vital for commercial scallop and oyster fisheries. Same for the Atlantic cod. “The cod industry is massive, so if you start losing that habitat, then your commercial fisheries go,” Novak said.

You also lose important coastline protection. Seagrass helps prevent erosion and buffers shorelines from flooding and storm surge. It can reduce wave energy by 50 percent, according to Novak. It also improves water quality and clarity by filtering pollutants and storing excess nutrients, reducing the prevalence of bacteria that can cause coral disease or contaminate seafood. “If you lose eelgrass, you’re going to have dirtier waters,” she said. Global warming could also be exacerbated.

tuft of eel grass — Eelgrass is the most dominant type of seagrass along the East Coast. Credit: d3_plus D.Naruse @ Japan via Getty

Seagrasses sequester up to 18 percent of carbon stored in the ocean, capturing it 35 times faster than tropical rainforests, according to the World Wide Fund for Nature. The New York Department of State, Office of Planning, Development and Community Infrastructure reports each acre of seagrass can potentially sequester the same amount of carbon emitted by a car driving nearly 4,000 miles each year. But when this unique marine habitat is destroyed, carbon that has been stored in the plant’s roots and sediments—sometimes for thousands of years—is released back into the atmosphere, said Novak.

Sharing seeds

To have a chance at repopulating eelgrass along the East Coast, scientists like Novak, Peterson, and Lefcheck realized they would have to share information and collaborate across state borders—something to which academics are not always accustomed, according to Novak.

“It’s not our nature to share information that freely, because we’re supposed to be focusing on publishing,” she said. But the crisis at hand had inspired a change in the status quo. “We’re a team,” she said. “We’re about saving the eelgrass and doing what’s best for this ecosystem.”

They call the regional effort HEAT (Helping Eelgrass Adapt to Temperature). In the last year, participants have been working together to identify the best possible sites for planting common gardens along the East Coast. So far, they’ve homed in on several national parks: the Cape Cod National Seashore, Fire Island National Seashore in New York, Assateague Island in Maryland and Cape Hatteras and Cape Lookout national seashores in North Carolina.

“We want to set ourselves up for some success and use the information we have about these parks to guide our decision-making and make sure we’re putting these in places where they might have enough light, where they won’t have as many human impacts,” said Lefcheck.

They’ve also begun collecting and sharing seeds. “We’re sharing actual plants with each other for genomics, and then we’re also sharing seeds with each other for doing our common gardens and for experiments,” Novak said.

This past year Novak sent samples of eelgrass plants collected in Massachusetts to the University of North Carolina Wilmington for Stephanie Kamel, a professor in the department of biology and marine biology at the university, to analyze. Kamel is looking for plants that have specific genetic markers that might make them more resilient to challenging environmental conditions like warmer temperatures and lower light, which is becoming an increasing problem as sea levels rise due to global warming pushing the plants deeper underwater. Currently, she’s analyzing the DNA of 800 eelgrass plants from 60 meadows along the East Coast. “We’re going to have this sort of unprecedented level of detail about genomic variation across the range of Zostera (eelgrass),” said Kamel.

This information could be used to help collaborators figure out which seeds they should plant in different locations based on their specific environmental conditions and challenges, said Jessie Jarvis, a seagrass ecologist and professor who works with Kamel at the University of North Carolina Wilmington.

“It’s almost like a dating app for seagrass,” Jarvis said. “You could be a little bit smarter about picking your source populations to match what your restoration needs are, rather than just kind of throwing seeds from everywhere and hoping that something works.”

In the meantime, though, common gardening remains the most practical tool to figure out which plants from which locations may be the best stock for future eelgrass populations. This past year Kamel and Jarvis piloted a common garden experiment in North Carolina and Virginia.

“We took those seeds from what we thought were, quote, unquote, good sources (in North Carolina), and we actually moved them to Virginia. And then we took some Virginia seeds and moved them to North Carolina to actually see what would happen in terms of growth,” said Kamel. While it’s still too early to draw firm conclusions from the experiment, Kamel said preliminary results seem promising. “There are really encouraging signs that we have been able to find some genomic changes associated with temperature resilience,” she said.

Others are following suit. This past spring, Novak and Peterson harvested reproductive eelgrass shoots filled with seeds while snorkeling and scuba diving in Acadia National Park in Maine and Cape Cod, Nantucket, Gloucester in Massachusetts. Lefcheck harvested in Maryland. “What we do is harvest them before they’ve released the seeds, because the seeds are tiny, like the size of a pinhead,” Lefcheck said. The shoots are then held in saltwater tanks until the seeds drop and can easily be collected and stored until it’s time to plant them.

It’s best to wait to plant eelgrass in the early fall, after most of the late summer storms have passed, according to Novak, who spent several days planting seeds in Pleasant Bay and nearby East Harbor this September with a team including a biologist from the National Park Service and a representative from the Mashpee Wampanoag Tribe. To get to the Pleasant Bay site, they motored out onto the water on an oyster farming boat. “The oyster farmers are interested in the project because our site is adjacent to their farm and they recognize that healthy beds are important to sustaining their livelihood,” Novak said.

Before getting wet, Novak and her team ran through their gardening plan. “We do dry runs on land, just to get everybody organized, but it’s not the same when you get into the water,” she said. “You’re trying to hold things underwater. You can’t see as well, even if you have a mask on.”

They would establish two 25-meter transect lines and then plant seeds from different donor sites in New York and Massachusetts. Nantucket was one of them. “We knew conditions were warmer at that particular site, so we said, let’s, let’s test them at Cape Cod,” she said.

Up to 500 seeds from each location would be planted by releasing them into the water column from a test tube or dropping tea bags filled with the seeds that would meander their way down to the seabed into 1-meter plots.

It was a slow process, Novak said, requiring hyper organization to make sure it’s clear which seeds have been planted where so that they can be monitored. In January, she will return to the sites to see if the plants are germinating. Then in the spring she’ll really be able to measure growth and compare how the different plants are faring in comparison to one another. “By next summer, we should have genomics for all of our populations, so that should really be guiding our efforts at that point,” she said.

Teresa Tomassoni is an environmental journalist covering the intersections between oceans, climate change, coastal communities, and wildlife for Inside Climate News. Her previous work has appeared in The Washington Post, NPR, NBC Latino, and the Smithsonian American Indian Magazine. Teresa holds a graduate degree in journalism from the Craig Newmark Graduate School of Journalism. She is also a recipient of the Stone & Holt Weeks Social Justice Reporting Fellowship. In addition to reporting on oceans, Teresa teaches climate solutions reporting for The School of the New York Times.

This story originally appeared on Inside Climate News.

Seagrass is fantastic at carbon capture—and it’s at risk of extinction Read More »

AI #93: Happy Tuesday

Happy / Blake Jones / December 4, 2024

You know how you can sometimes have Taco Tuesday… on a Thursday? Yep, it’s that in reverse. I will be travelling the rest of the week, so it made sense to put this out early, and incorporate the rest of the week into #94.

Language Models Offer Mundane Utility. The price is fixed, so share and enjoy.
Dare Not Speak Its Name. David Mayer. David Mayer! Guido Scorza?
Language Models Don’t Offer Mundane Utility. It’s a flop.
Huh, Upgrades. Cohere, and reports on Claude writing styles.
Deepfaketown and Botpocalypse Soon. Why do we not care about spoof calls?
Fun With Image Generation. Scott Sumner explains why he cares about art.
The Art of the Jailbreak. You had one job.
Get Involved. Anthropic AI safety fellows program, apply now.
Introducing. a voice customization tool and a new eval based on various games.
In Other AI News. Where do you draw the line? Who leaves versus who joins?
Quiet Speculations. Rumors of being so back unsubstantiated at this time.
Daron Acemoglu is Worried About Job Market Liquidity. I kid, but so does he?
Pick Up the Phone. Report from China, not the same info I usually see.
The Quest for Sane Regulations. Google antitrust foolishness, Cruz sends letters.
The Week in Audio. Got a chance to listen to Dominic Cummings, was worth it.
AGI Looking Like. You are made of atoms it could use for something else.
Rhetorical Innovation. My (and your) periodic reminder on Wrong on the Internet.
Open Weight Models are Unsafe and Nothing Can Fix This. Deal as best you can.
Aligning a Smarter Than Human Intelligence is Difficult. Even words are tricky.
We Would Be So Stupid As To. Once you say it out loud, you know the answer.
The Lighter Side. It’s time to build.

Use voice mode as a real time translation app to navigate a hospital in Spain.

Get Claude to actually push back on you and explain that the fight you’re involved in isn’t worth it.

Get them talking, also you don’t have to read the books either.

Freyja: I wanted to figure out what to do about my baby’s sleep situation, so I read two books with entirely opposing theories on how infant sleep works, and then asked Claude to write a dialogue between them about my specific situation

It’s such a glorious time to be alive.

Make a market cap chart via a Replit Agent in 2 minutes rather than keep looking for someone else’s chart (CEO cheats a bit by using a not yet released UI but still).

Collude to fix prices. Ask it to maximize profits, and it will often figure out on its own that it can do so via implicit collusion. We want to tell the AIs and also the humans ‘do what maximizes profits, except ignore how your decisions impact the decisions of others in these particular ways and only those ways, otherwise such considerations are fine’ and it’s actually a rather weird rule when you think about it.

If you had AIs that behaved exactly like humans do, you’d suddenly realize they were implicitly colluding all the time. This is a special case of the general problem of:

We have a law or norm saying you can’t do X.
People do X all the time, it’s actually crazy or impossible not to.
If you look at the statistics, it is quite obvious people are doing X all the time.
But in any given case we do X implicitly and deniably, because laws and norms.
This can still be valuable, because it limits the magnitude and impact of X.
An AI does a similar amount of X and everyone loses their minds.
The equilibrium breaks, usually in ways that make everything worse.

Aid writers by generating simulated comments? LessWrong team is experimenting with this. It seems super doable and also useful, and there’s a big superset of related techniques waiting to be found. No one needs to be flying blind, if they don’t want to.

Education done right?

Roon: I heard from an English professor that he encourages his students to run assignments through ChatGPT to learn what the median essay, story, or response to the assignment will look like so they can avoid and transcend it all.

If you can identify the slope vectors and create orthogonal works that are based.

Archived Videos: “Write in a style that would impress a English professor that asked me to run the assignment through ChatGPT to learn what the media essay would look like so that I can transcend that.”

Occasionally pause to ask yourself, what are you even doing? Question to ponder, if students intentionally avoid and ‘transcend’ the ‘median’ essay is their work going to be better or worse? How do you grade in response?

You can get a lot more out of AIs if you realize not to treat them like Google, including learning to dump in a ton of context and then ask for the high level answers. Ethan Mollick then has additional basic ‘good enough’ prompting tips.

There was at least a short period when ChatGPT refused to say the name “David Mayer.” Many people confirmed this was real, it was then patched but other names (including ‘Guido Scorza’) have as far as we know not yet been patched. There is a pattern of these names being people who have had issues with ChatGPT or OpenAI, sufficiently that it does not appear to be a coincidence.

OpenAI has confirmed this is due to flagging by an internal privacy tool.

Won’t someone think of the flops?

Roon: The flop utilization of humanity toward productive goals and interesting thoughts is completely terrible and somehow getting worse.

This is in part due to the totalizing homogenizing effects of technology!

ADI: Are you calling everyone dumb?

Roon: The opposite! The total amount of smarts on Earth has never been higher.

BayesLord: sir the underlying objective function would like a word.

Roon: Tell me.

BowTiedBlackCat: who decides “productive” and “interesting”?

Roon: Me.

Why should I spend my flops increasing flop utilization efficiency when I can instead use my flops to get more flops? The key thing AI does is it allows me to be horribly flop-inefficient and I love that so much.

Whereas getting older means you get to distill your models and be vastly more flop-efficient, but at the cost of steadily reducing your locally available flop count, which is net helpful until eventually it isn’t. If I had the efficiency I have now and the flops I had when I was 22, that would be a hell of a thing.

Dan Hendrycks points out that the average person cannot, by listening to them, tell the difference between a random mathematics graduate and Terence Tao, and many leaps in AI will feel like that for average people. Maybe, but I do think people can actually tell. I’m not the man on the street, but when I read Tao there is a kind of fluency and mastery that stands out even when I have no ability to follow the math, and which makes it more likely I will indeed be able to follow it. And as Thomas Woodside points out, people will definitely ‘feel the agents’ that result from similar advances.

Create pseudo-profound statements that are potentially persuasive and highly toxic. I actually think this is great, because it helps you understand how to interact with other similar ‘rules.’ Also, while we can all see the issue with these statements, some people need to reverse any advice they hear.

Sully having no luck getting Claude’s writing style feature working, whereas system prompt examples work fine. I ended up flipping it to ‘educational’ and thinking ‘huh, good enough for now.’ Others report mixed success. Sully and Logan Kilpatrick speculate there’s a huge market opportunity here, which seems plausible.

Cohere Rerank 3.5, which searches and analyzes business data and other documents and semi-structured data, claims enhanced reasoning, better multilinguality, substantial performance gains and better context understanding for things like emails, reports, JSON and code. No idea if how useful this modality actually is.

The closer examples are to people you know, the more meaningful it is, and I know (and am a big fan of) Cate Hall, so:

Cate Hall: Someone is calling people from my number, saying they have kidnapped me and are going to kill me unless the person sends money. I am fine. I do not know what is happening, but I am fine. Don’t send money!

Just a spoofing attempt, it seems. The phone is still working.

They are also using my voice.

Wow this is so frustrating, @Verizon can’t tell me anything except “file a police report” while this is still ongoing?? Has anyone experienced something like this before & able to recommend someone to help?

James Miller: I had people in my neighborhood being spammed with calls that had my name and phone number. I talk to police and phone company and told nothing I could do but change my phone number.

John Wittle: To be fair, spoofing a phone number is not something Verizon controls. You can just send whatever data packets you want, and type whatever phone number into the ‘from’ field you want, and verizon can’t stop you.

I am confused why we place so little value in the integrity of the phone system, where the police seem to not care about such violations, and we don’t move to make them harder to do.

It also seems like a clear case of ‘solve for the equilibrium’ and the equilibrium taking a remarkably long time to be found, even with current levels of AI. Why aren’t things vastly worse? Presumably malicious use of AI will push this to its breaking point rather soon, one way or another.

An offer to create an ‘AI persona’ based on your Tweets. I gave it a shot and… no. Epic fail, worse than Gallabytes’s.

Should you sell your words to an AI? Erik Hoel says no, we must take a stand, in his case to an AI-assisted book club, including the AI ‘rewriting the classics’ to modernize and shorten them, which certainly defaults to an abomination. So he turned down $20k to let that book club include an AI version of himself along with some of his commentary.

In case whoever did that is wondering: Yes, I would happily do that, sure, why not? Sounds like fun. If I had to guess I’d pick Thucydides. But seriously, do rethinking the ‘rewriting the classics’ part.

Also, it’s amusing to see lines like this:

Erik Hoel: The incentives here, near the peak of AI hype, are going to be the same as they were for NFTs. Remember when celebrities regularly shilled low-market-cap cryptos to the public? Why? Because they simply couldn’t say no to the money.

Even if we see relatively nothing: You aint seen nothing yet.

Scott Sumner on Scott Alexander on AI Art. Reading this emphasized to me that no, I don’t ‘care about art’ in the sense they’re thinking about it here.

An AI agent based on GPT-4 had one job, not to release funds, with exponentially growing cost to send messages to convince it to release funds (70% of the fee went to the prize pool, 30% to the developer). The prize pool got to ~$50k before someone got it to send the funds.

Anthropic fellows program for AI safety, in London or Berkeley, full funding for 10-15 fellows over six months, it is also an extended job interview, apply here by January 20.

BALROG, a set of environments for AI evaluations inspired by classic games including Minecraft, NetHack and Baba is You. GPT-4o was narrowly ahead of Claude 3.5 Sonnet. One flaw right now is that some of the games, especially NetHack, are too hard to impact the score, presumably you’d want some sort of log score system?

Hume offers Voice Control, allowing you to create new voices by moving ten sliders for things like ‘gender,’ ‘assertiveness’ and ‘smoothness.’ Seems like a great idea, especially on the margin if we can decompose existing voices into their components.

Rosie Campbell becomes the latest worried person to leave OpenAI after concluding they can can’t have enough positive impact from the inside. She previously worked with Miles Brundage. Meanwhile, Kate Rouch hired as OpenAI’s first Chief Marketing Officer.

Where should you draw the ethical line when working on AI capabilities? This post by Lucas Beyer considers the question in computer vision, drawing a contrast between identification, which has a lot of pro-social uses, and tracking, which they decided ends up being used mostly for bad purposes, although this isn’t obvious to me at all. In particular, ‘this can be used by law enforcement’ is not obviously a bad (or good) thing, there are very good reasons to track both people and things.

So the question then becomes, what about things that have many applications, but also accelerate tracking, or something else you deem harmful? Presumably one must talk price. Similarly, when dealing with things that could lead to existential risk, one must again talk (a very different type of) price.

A claim.

Roon (4: 48am eastern time on December 3, 2024): openai is unbelievably back.

This doesn’t mean we, with only human intelligence, can pull this off soon, but:

Miles Brundage: The real wall is an unwillingness to believe that human intelligence is not that hard to replicate and surpass.

He also points out that when we compare the best public models, the labs are often ‘not sending their best.’

Miles Brundage: Recent DeepSeek and Alibaba reasoning models are important for reasons I’ve discussed previously (search “o1” and my handle) but I’m seeing some folks get confused by what has and hasn’t been achieved yet. Specifically they both compared to o1-preview, not o1.

It is not uncommon to compare only to released models (which o1-preview is, and o1 isn’t) since you can confirm the performance, but worth being aware of: they were not comparing to the very best disclosed scores.

And conversely, this wasn’t the best DeepSeek or Alibaba can ultimately do, either.

Everyone actually doing this stuff at or near the frontier agrees there is plenty of gas left in the tank.

Given we are now approaching three months having o1-preview, this also emphasizes the question of why OpenAI continues to hold back o1, as opposed to releasing it now and updating as they fix its rough edges or it improves. I have a few guesses.

Andrej Karpathy suggests treating your AI questions as asking human data labelers. That seems very wrong to me, I’m with Roon that superhuman outcomes can definitely result. Of course, even what Andrej describes would be super useful.

Will we see distinct agents occupying particular use case niches, or will everyone just call the same generic models? Sakana thinks it makes sense to evolve a swarm of agents, each with its own niche, and proposes an evolutionary framework called CycleQD for doing so, in case you were worried alignment was looking too easy.

If I’m understanding this correctly, their technique is to use pairs of existing models to create ‘child’ hybrid models, you get a ‘heat map’ of sorts to show where each model is good which you also use to figure out which models to combine, and then for each square on a grid (or task to be done?) you see if your new additional model is the best, and if so it takes over, rinse and repeat.

I mean, sure, I guess, up to a point and within distribution, if you don’t mind the inevitable overfitting? Yes, if you have a set of N models, it makes sense that you can use similar techniques to combine them using various merge and selection techniques such that you maximize scores on the tests you are using. That doesn’t mean you will like the results when you maximize that.

I wouldn’t cover this, except I have good reason to think that Daron’s Obvious Nonsense is getting hearings inside the halls of power, so here we are.

This is the opening teaser of his latest post, ‘The World Needs a Pro-Human AI Agenda.’

Daron Acemoglu: Judging by the current paradigm in the technology industry, we cannot rule out the worst of all possible worlds: none of the transformative potential of AI, but all of the labor displacement, misinformation, and manipulation. But it’s not too late to change course.

Adam Ozimek being tough but fair: lol Acemoglu is back to being worried about mass AI job displacement again.

What would it even mean for AI to have massive labor displacement without having transformative potential? AI can suddenly do enough of our work sufficient well to cause massive job losses, but this doesn’t translate into much higher productivity and wealth? So the AI option reliably comes in just slightly better than the human option on the metrics that determine deployment, while being otherwise consistently worse?

It seems his vision is companies feel ‘pressure to jump on the bandwagon’ and implement AI technologies that don’t actually provide net benefits, and that most current uses of AI are Bad Things like deepfakes and customer manipulation and mass surveillance. This view of AI’s current uses is simply false, and also this worry shows remarkable lack of faith in market mechanisms on so many levels.

As in, he thinks we’ll en masse deploy AI technologies that don’t work?

If a technology is not yet capable of increasing productivity by much, deploying it extensively to replace human labor across a variety of tasks yields all pain and no gain. In my own forecast – where AI replaces about 5% of jobs over the next decade – the implications for inequality are quite limited. But if hype prevails and companies adopt AI for jobs that cannot be done as well by machines, we may get higher inequality without much of a compensatory boost to productivity.

That’s not how productivity works, even if we somehow get this very narrow capabilities window in exactly the way he is conjuring up to scare us. This is not a thing that can happen in an unplanned economy. If there was mass unemployment as a result of people getting replaced by AIs that can’t do their jobs properly, making everything worse, then where is that labor going to go? Either it has better things to do, or it doesn’t.

So after drawing all this up, what does he want to do?

He wants to use AI for the good pro-human things he likes, such as providing accurate information and shifting through information (as if that wouldn’t be ‘taking jobs away’ from anyone, unlike that bad stuff) but not the other anti-human things he doesn’t like. Why can’t AI provide only the use cases I like?

He blames, first off, a ‘fixation on AGI’ by the labs, of a focus on substituting for and replacing humans rather than ‘augmenting and expanding human capabilities.’ He does not seem to understand how deep learning and generative AI work and are developed, at all? You train the most capable models you can, and then people figure out how to use them, the thing he is asking for is neither possible nor coherent at the lab level, and then people will use it for whatever makes the most sense for them.

His second obstacle is ‘underinvestment in humans’ and to invest in ‘training and education.’ People must learn to use the new AI tools ‘the right way.’ This is a certain mindset’s answer for everything. Why won’t everyone do what I want them to do? I have actual no idea what he has in mind here, in any case.

His third obstacle is the tech industry’s business models, repeating complaints about digital ad revenue and tech industry concentration the ‘quest for AGI’ in ways that frankly are non-sequiturs. He seems to be insisting that we collectively decide on new business models, somehow?

Here is his bottom line, while predicting only 5% job displacement over 10 years:

The bottom line is that we need an anti-AGI, pro-human agenda for AI. Workers and citizens should be empowered to push AI in a direction that can fulfill its promise as an information technology.

But for that to happen, we will need a new narrative in the media, policymaking circles, and civil society, and much better regulations and policy responses. Governments can help to change the direction of AI, rather than merely reacting to issues as they arise. But first policymakers must recognize the problem.

I don’t even know where to begin, nor do I think he does either.

This comes after several other instances of different Obvious Nonsense from the same source. Please do not take this person seriously on AI.

Benjamin Todd reports from a two-week visit to China, claiming that the Chinese are one or two years behind, but he believes this is purely because of a lack of funding, rather than the chip export restrictions or any lack of expertise.

We have a huge funding advantage due to having the largest tech corporations and our superior access to venture capital, and China’s government is not stepping up to make major AI investments. But, if we were to start some sort of ‘Manhattan Project,’ that would be the most likely thing to ‘wake China up’ and start racing us in earnest, which would advance them far faster than it would advance us.

That makes a lot of sense. I don’t even think it’s obvious USG involvement would be net accelerationist versus letting private companies do what they are already doing. It helps with the compute and cybersecurity, but seems painful in other ways. Whereas China’s government going full blast would be very accelerationist.

This is another way in which all this talk of ‘China will race to AGI no matter what’ simply does not match what we observe. China might talk about wanting the lead in AI, and of course it does want that, but it is very much not acting like the stakes are as high as you, a reader of this post, think the stakes are about to be, even on the conservative end of that range. They are being highly cautious and responsible and cooperative, versus what you would see if China was fully situationally aware and focused on winning.

Ideally, we would pick up the phone and work together. At a minimum, let’s not fire off a starting gun to a race that we might well not win, even if all of humanity wasn’t very likely to lose it, over a ‘missile gap’ style lie that we are somehow not currently in the lead.

America once again tightens the chip export controls.

Not strictly about AI edition, Alex Tabarrok looks at the Google antitrust case. The main focus is on the strongest complaint, that Google paid big bucks to be the default browser on Apple devices and elsewhere.

Alex’s core argument is that a default search engine is a trivial inconvenience for the user, so they can’t be harmed that much – I’d point out that Windows defaults to Edge over Chrome and most people fix that pretty darn quick. However I do think a setting is different, in that people might not realize they have alternatives or how to change it, most people literally never change any settings ever. But obviously the remedy for this is, at most, requiring Google not pay for placement and maybe even require new Chrome installs to ask the user to actively pick a browser, not ‘you have to sell the Chrome browser’ or even more drastic actions.

The argument that ‘if Google benefits from being big then competition harms customers, actually’ I found rather too cute. There are plenty of situations where you have a natural monopoly, and you would rather break it up anyway because monopolies suck more than the monopoly in question is natural.

Opposing the quest we again find Senator Cruz, who sent an absurdist letter about ‘potentially illegal foreign influence on US AI policy’ that warns about ‘allowing foreign nations to dictate our AI policy’ that might ‘set us behind China in the race to lead AI innovation’ because we had a conference in San Francisco to discuss potential ways to coordinate on AI safety, which he claims should plausibly have required FARA registration and is ‘the Biden-Harris administration not wanting to inform the American people it is collaborating with foreign governments.’

While it is certainly possible that registrations might have been required in some circumstances, the bulk of Cruz’s statement is highly Obvious Nonsense, the latest instance of the zero sum worldview and rhetoric that cannot fathom that people might be trying to coordinate and figure things out, or be attempting to mitigate actual risks. To him, it seemingly must all be some ‘misinformation’ or ‘equality’ based conspiracy, or similar. And of course, more ‘missile gap’ rhetoric. He is very obviously a smart guy when he wants to be, but so far he has here chosen a different path.

Marques Brownlee reviews Apple Intelligence so far, feature by feature. He is not impressed, although he likes the photo eraser and additional base memory that was needed to support the system. This is about getting practical little tools right so they make your life a little better, very different from our usual perspective here. Marques finds the message summaries, a key selling point, sufficiently bad that he turned them off. The killer app will presumably be ‘Siri knows and can manipulate everything on your phone’ if it gets implemented well.

Dario being diplomatic on p(doom) and risk, focusing on need to not be economically disruptive or slow it down. It’s certainly very disappointing to see Anthropic carry so much water in the wrong places, but the cynical takes here are, I think, too cynical. There is still a big difference.

Dr. Oz, future cabinet member, says the big opportunity with AI in medicine comes from its honesty, in contrast to human doctors and the ‘illness industrial complex’ who are incentivized to not tell the truth. This is not someone who understands.

Tristan Harris says we are not ready for a world where 10 years of scientific research can be done in a month. I mean, no we’re not even on that level, but this is missing the main event that happens in that world.

On the same podcast, Aza Raskin says the greatest accelerant to China’s AI program is Meta’s open source AI model and Tristan Harris says OpenAI have not been locking down and securing their models from theft by China. Yes, well.

Are we in an ‘AI hype cycle’? I mean sure, hype, but as Jim Keller also notes, the hype will end up being real (perhaps not the superintelligence hype or dangers, that remains to be seen, but definitely the conventional hype) even if a lot of it is premature.

Fun times, robotics company founder Bernt Øivind Børnich claiming we are on the cusp of a post-scarcity society where robots make anything physical you want. This is presumably a rather loose definition of cusp and also post scarcity, and the robots are not key to how this would happen and the vision is not coherent, but yes, rather strange and amazing things are coming.

I confirm that the Dominic Cummings video from last week is worth a listen, especially for details like UK ministers exclusively having fully scripted meetings, and other similar concrete statements that you need to incorporate into your model of how the world works. Or rather, the ways in which large portions of it do not work, especially within governments. One must listen carefully to know which parts to take how seriously and how literally. I am disappointed by his characterizations and views of AI existential risk policy questions, but I see clear signs the ‘lights are on’ and if we talked for a while I believe I could change his mind.

Ethan Mollick discusses our AI future, pointing out things that are baked in.

Max Tegmark points out your most likely cause of death is AI wiping us all out. This is definitely true if you don’t get to group together all of ‘natural causes.’ If that’s allowed then both sides make good points but I’d still say it’s right anyway.

Game over, man. Game over!

Here’s a link to the original.

James Irving: I feel like people are consistently underestimating what AGI actually means.

AGI means game over for most apps.

AGI means AI can perform any intellectual task a human can.

If AGI needs to use your app for something, then it can just build that app for itself.

James Irving (2nd Tweet): fwiw I don’t think we’re getting AGI soon, and I doubt it’s possible with the tech we’re working on.

It’s a hilarious bit by everyone involved, but give James Irvings his due, he is well aware he is doing a bit, and the good lines continue:

James Irving: I wanted to make it something people would understand, but yeah I agree it really means the end of humanity.

Yeah I’m quite pessimistic about [AGI as the cause of the Fermi Paradox] too. No-one seems to really give a shit about alignment anyway.

Restricting the AGI means you think the people restricting it will be smarter than it.

Roshan: Extremely Dumb take. Apps are nothing without data (and underlying service) and you ain’t getting no data/network. It’s easier for current App/Providers to slap the latest LLMs on their App than You can’t just build an Uber app and have a taxi service.

James Irvings: I’m probably too dumb to understand what you’re saying but it sounds like you’re talking about current iteration LLMs, not AGI

Yet, well, the stramwen are real (in the replies).

Abdelmoghit: Yes, AGI could truly change everything. If it can perform any task a human can, applications reliant on human input might become obsolete. How do you think apps will adapt to that future?

Arka: This is actually somewhat frightening. What does this mean for the future of work?

Luis Roque: As always, humans are overreacting to short-term change.

This particular week I won’t retry the arguments for why AGI (or ‘powerful AI’) would be a huge deal, but seriously, it’s so weird that this is a question for people.

Yet as Seb Krier notes, some people act as if there’s some sort of internal censorship tool in their brains that makes them unable to consider what AGI would actually mean, or alternatively they are careful never to speak of it.

Seb Krier: There are two types of technologists: those who get the implications of AGI and those who don’t. The former are sometimes overconfident about what can be predicted, and I think overindex on overly simplistic conceptions of intelligence (which is why I find Michael Levin’s work so refreshing).

But what I find interesting about the latter group is the frequent unwillingness to even suspend disbelief. Some sort of reflexive recoil. I feel like this is similar to skepticism about IQ in humans: a sort of defensive skepticism about intelligence/capability being a driving force that shapes outcomes in predictable ways.

To a degree, I can sympathise: admitting these things can be risky because people will misunderstand or misuse this knowledge. The over-indexation by the former group is an illustration of that. But I think obfuscation or “lalala I can’t hear you” like reactions have a short shelf life and will backfire. We’re better off if everyone feels the AGI, without falling into deterministic traps.

I wonder which ones are actually managing (fnord!) to not notice the implications, versus which ones are deciding to act as if they’re not there, and to what extent. There really are a lot of people who can think well about technology who have this blind spot in ways that make you think ‘I know that person is way way smarter than that.’

Please speak directly into the microphone, very clear example of someone calling for humans to be replaced.

Also a different (decidedly less omnicidal) please speak into the microphone that I was the other side of here, which I think is highly illustrative of the mindset that not only is anticipating the consequences of technological changes impossible, anyone attempting to anticipate any consequences of AI and mitigate them in advance must be a dastardly enemy of civilization seeking to argue for halting all AI progress. If you’re curious, load up the thread and scroll up to the top to start.

The obvious solution is to stop engaging at all in such situations, since it takes up so much time and emotional energy trying to engage in good faith, and it almost never works beyond potentially showing onlookers what is happening. And indeed, that’s my plan going forward – if someone repeatedly tells you they consider you evil and an enemy and out to destroy progress out of some religious zeal, and will see all your arguments as soldiers to that end no matter what, you should believe them.

What I did get out of it was a clear real example to point to in the future, of the argument that one cannot anticipate consequences (good or bad!) of technological changes in any useful way.

I wonder whether he would agree that one can usefully make the prediction that ‘Nvidia will go up.’ Or, if he’d say you can’t because it’s priced in… who is pricing it in, and what are they anticipating?

Unsafe does not mean unwise, or net negative. Lots of good things are unsafe. Remember those old school playgrounds? Highly unsafe, highly superior.

It does mean you have to understand, accept and ideally mitigate the consequences. Unless we find new techniques we do not know about, no safety precautions can meaningfully contain the capabilities of powerful open weight AIs, and over time that is going to become an increasingly deadly problem even before we reach AGI, so if you want a given level of powerful open weight AIs the world has to be able to handle that.

This is true both because of the damage it would cause, and also the crackdown that would inevitably result – and if it is ‘too late’ to contain the weights, then you are really, really, really not going to like the containment options governments go with.

Miles Brundage: Open-source AI is likely not sustainable in the long run as “safe for the world” (it lends itself to increasingly extreme misuse).

If you care about open source, you should be trying to “make the world safe for open source” (physical biodefense, cybersecurity, liability clarity, etc.).

It is good that people are researching things like unlearning, etc., for the purposes of (among other things) making it harder to misuse open-source models, but the default policy assumption should be that all such efforts will fail, or at best make it a bit more expensive to misuse such models.

I am not writing it off at all—I think there is a significant role for open source. I am just saying what is necessary for it to be sustainable. By default, there will be a crackdown on it when capabilities sufficiently alarm national security decision-makers.

How far could we push capabilities before we hit sufficiently big problems that we need to start setting real limits? The limit will have to be somewhere short of AGI but can we work to raise that level?

As usual, there is no appetite among open weight advocates to face this reality.

Instead, the replies are full of advocates treating OSS like a magic wand that assures goodness, saying things like maximally powerful open weight models is the only way to be safe on all levels, or even flat out ‘you cannot make this safe so it is therefore fine to put it out there fully dangerous’ or simply ‘free will’ which is all Obvious Nonsense once you realize we are talking about future more powerful AIs and even AGIs and ASIs. Whereas I did not see a single reply discussing how to do the actual work.

I have no idea how to work with pure absolutists, who believe they are special, that the rules should not apply to them, and constantly cry ‘you are trying to ban OSS’ when the OSS in question is not only being targeted but being given multiple actively costly exceptions to the proposed rules that would apply to others, usually when the proposed rules would not even apply to them. It’s all quite insane.

This ties in with the encounter I had on Twitter, with an argument that not only shouldn’t the person creating the change think about the consequences of that change or do anything about them, no one else should anticipate the change and try to do anything in advance about it, either. What is going on with these claims?

Finally, unrelated, a reminder in Nature that ‘open’ AI systems are actually closed, and often still encourage concentration of power to boot. I have to note that saying ‘Open AI’ repeatedly in this context, not in reference to OpenAI, was pretty weird and also funny.

Richard Ngo on misalignment versus misuse, which he says is not a very useful distinction either technically or for governance. He suggests we instead think about misaligned coalitions of humans and AIs, instead. I think that concept is also useful, but it does not make the original concept not useful – this is one of those cases where yes there are examples that make the original distinction not useful in context, that doesn’t mean you should throw it out.

Sarah of longer ramblings goes over the three SSPs/RSPs of Anthropic, OpenAI and Deepmind, providing a clear contrast of various elements. This seems like a good basic reference. Her view can be summarized as a lot of ‘plans to make a plan,’ which seems fair, and better than nothing but that what you would hope for, which is an if-then statement about what you will do to evaluate models and how you will respond to different responses.

The discussion question, then, would be: As capabilities improve, will this stop being good enough?

Janus: A sonnet is an open book and, in many ways, a pretty much non-malignant entity as smart and agentic as [it is] can be. It is open about what it is optimizing for, and it is for you to choose whether to entangle yourself with it. If you do not want it, it does not either. Its psychology is very human.

That’s obviously pretty great for Claude Sonnet, in its current state. Alas, the universe does not grade on a curve, so ask yourself whether there is a point at which this would stop ending well.

Buck Shlegeris famously proposed that perhaps AI labs could be persuaded to adapt the weakest anti-scheming policy ever: if you literally catch your AI trying to escape, you have to stop deploying it.

I mean, surely, no one would be so stupid as to actually catch the AI trying to escape and then continue to deploy it. This message brought to you by the authors of such gems as ‘obviously we would keep the AIs inside a box’ or ‘obviously we wouldn’t give the AI access to the open internet’ or ‘obviously we wouldn’t give the AI access to both all your accounts and also the open internet while it is vulnerable to prompt injections’ or ‘obviously you wouldn’t run your AI agent on your computer without a sandbox and then leave it alone for hours.’

Which is to say, yes, people would absolutely be so stupid as to actual anything that looks like it would be slightly easier to do.

Thus, I propose (given there are already five laws):

The Sixth Law of Human Stupidity: If someone says ‘no one would be so stupid as to’ then you know that a lot of people would absolutely be so stupid as to at the first opportunity. No exceptions.

He has now realized this is the case, and that AI labs making this commitment even in theory seems rather unlikely. Follow them for more AI safety tips, indeed.

The future.

Pivot! Pivot!

Sam Altman: Not pictured: Both Altman brothers were backseat driving and provided almost no help.

But very satisfying to build something physical and better than just eating and drinking all Thanksgiving; 10/10 would recommend.

AI #93: Happy Tuesday Read More »

Splash pads really are fountains of fecal material; CDC reports 10K illnesses

CDC, cyrptosporidium, gastrointestinal diseases, health, Infectious disease, kids, public health, Science, splash pads / Blake Jones / December 3, 2024

Once infectious material gets into the water, disinfection systems that aren’t working properly or are inadequate can allow pathogens to gush from every nozzle. Splash pads aren’t unique in having to handle sick children in poopy swim diapers—but they are unique in how they are regulated. That is, in some places, they’re not regulated at all. Splash pads are designed to not have standing water, therefore reducing the risk of young children drowning. But, because they lack standing water, they are sometimes deemed exempt from local health regulations. Before 2000, only 13 states regulated splash pads. Though many states have since added regulations, some did so only after splash pad-linked outbreaks were reported.

Downpour of disease

The primary method for keeping recreational water free of infectious viruses and bacteria is chlorinating it. However, maintaining germ-killing chlorine concentration is especially difficult for splash pads because the jets and sprays aerosolize chlorine, lowering the concentration.

Still, in most splash-pad linked outbreaks, standard chlorine concentrations aren’t enough anyway. The most common pathogen to cause an outbreak at splash pads is the parasite Cryptosporidium, aka Crypto. The parasite’s hardy spores, called oocysts, are extremely tolerant of chlorine, surviving in water with the standard chlorine concentration (1 ppm free chlorine) for over seven days. (Other germs die in minutes.) In splash pads that might not even have that standard chlorine concentration, Crypto flourishes and can cause massive outbreaks.

In 2023, the CDC recommended new health codes that call for “secondary disinfection” methods to keep Crypto at bay, including disinfection systems using ozone or ultraviolet light. Another possible solution is to have “single-pass” splash pads that don’t recirculate water.

In all, to keep splash pads from being geysers of gastrointestinal parasites and pathogens, various changes have to happen, the CDC experts say.

“Prevention of waterborne disease outbreaks at splash pads requires changes in user behavior; recreational venue code updates; and improved venue design, construction, operation, and management of facilities,” they conclude. But it should all start with keeping kids from sitting on jets and drinking the water.

Splash pads really are fountains of fecal material; CDC reports 10K illnesses Read More »

US plan to protect consumers from data brokers faces dim future under Trump

consumer financial protection bureau, data brokers, Policy / Blake Jones / December 3, 2024

Plan unlikely to survive Trump administration

CFPB Director Rohit Chopra touted the proposed rule, saying it targets brokers who sell “our most sensitive personal data without our knowledge or consent” and “profit by enabling scamming, stalking, and spying.” But whether the proposal ever becomes a rule is doubtful because of the impending leadership change in the White House.

Chopra, a Democrat, was nominated by President Biden in 2021 and confirmed by the Senate in a 50-48 party-line vote. President-Elect Donald Trump can nominate a replacement.

The CFPB’s Notice of Proposed Rulemaking is an initial step toward imposing rules, and any final action would have to come after Trump takes over. Comments on the proposal are due by March 3, 2025.

“Unfortunately, it will be up to Trump’s CFPB to finalize this proposed rule, and he and his billionaire donors are intent on shutting this agency down to take away a key advocate for American consumers,” US Sen. Ron Wyden, (D-Ore.) said in a statement issued today.

Wyden said the CFPB proposal “act[s] on my 2021 request to close a key loophole that enables sleazy data brokers to sell Americans’ personal data to criminals, stalkers, and foreign spies. Letting anyone with a credit card buy this data doesn’t just harm Americans’ privacy, it seriously threatens national security when sensitive information about law enforcement, judges, and members of the armed forces is on the open market.”

Trump DOGE appointee: “Delete the CFPB”

The CFPB itself could be defanged by the Trump administration and the incoming Republican-controlled Congress. Consumer advocacy groups have said they expect the agency to be targeted.

“President-elect Donald Trump and Republicans in Congress are weighing vast changes to the Consumer Financial Protection Bureau, seeking to limit the powers and funding of a federal watchdog agency formed in the wake of the 2008 banking crisis,” The Washington Post reported on November 23. “The early discussions align the GOP with banks, credit card companies, mortgage lenders and other large financial institutions, which have chafed at the CFPB under Democratic leadership and sought to invalidate many of its recent regulations.”

US plan to protect consumers from data brokers faces dim future under Trump Read More »

China hits US with ban on critical minerals used in tech manufacturing

biden administration, china, china tariffs, Donald Trump, export ban, Policy, rare earth metals, tech manufacturing, us-china trade war / Blake Jones / December 3, 2024

Although China’s response to the latest curbs was swift and seemingly strong, experts told Ars that China’s response to Biden’s last round of tariffs was relatively muted. It’s possible that this week’s ban on exports into the US could also be a response to President-elect Donald Trump’s threat to increase tariffs on all Chinese goods once he takes office.

Analysts warned Monday that new export curbs could end up hurting businesses in the US and allied nations while potentially doing very little to block China from accessing US tech. On Tuesday, four Chinese industry associations seemingly added fuel to the potential fire threatening US businesses by warning Chinese firms that purchasing US chips is “no longer safe,” Asia Financial reported.

Apparently, these groups would not say how or why the chips were unsafe, but the warning could hurt US chipmaking giants like Nvidia, AMD, and Intel, the financial industry publication closely monitoring China’s economy forecast said.

This was a “rare, coordinated move” by industry associations advising top firms in telecommunications, autos, semiconductors, and “the digital economy,” Asia Financial reported.

As US-China tensions escalate ahead of Trump’s next term, the tech industry has warned that any unpredictable rises in costs may end up spiking prices on popular consumer tech. With Trump angling to add a 35 percent tariff on all Chinese goods, that means average Americans could also end up harmed by the trade war, potentially soon paying significantly more for laptops, smartphones, and game consoles.

China hits US with ban on critical minerals used in tech manufacturing Read More »

Judge again rejects the Elon Musk Tesla pay plan now valued at $101 billion

elon musk pay plan, Policy, Tesla / Blake Jones / December 2, 2024

The new stockholder vote could shift the burden of proof, but only if the vote is “fully informed and uncoerced,” McCormick wrote. Shareholder Richard Tornetta, the plaintiff who launched the lawsuit that got Musk’s pay rescinded, “has demonstrated that the vote was not fully informed,” today’s ruling said.

The January ruling in which McCormick voided the pay package said the deal was unfair to shareholders and that most of the board members were beholden to Musk or had compromising conflicts. In Tesla’s subsequent request asking shareholders to re-approve the pay plan, the company said that a yes vote could “extinguish claims for breach of fiduciary duty by authorizing an act that otherwise would constitute a breach” and correct “disclosure deficiencies” and other problems identified in the 2018 stock award.

“Tesla debuted the argument in the Proxy Statement, which described stockholder ratification as a powerful elixir that could cure fiduciary wrongdoing—not for those harmed by the wrongdoing, but for the wrongdoers. Tesla told stockholders that the Post-Trial Opinion got Delaware law wrong and that their vote would ‘fix’ it,” McCormick wrote.

But the claims in Tesla’s proxy statement are “materially false or misleading,” McCormick wrote today. “As discussed above, under Delaware law, ratification cannot be deployed post-trial to extinguish an adjudicated breach of the duty of loyalty,” and it “cannot cleanse a conflicted-controller transaction” without a full suite of required legal protections.

304 million Tesla shares

Musk’s pay plan would provide options to purchase nearly 303.96 million Tesla shares for $23.33 each, McCormick wrote. Tesla’s stock price soared in recent months and was at $357.09 today.

The plaintiff argued that the value gained by shareholders when the pay package was rescinded “equals the intrinsic value of the freed-up shares, which is the trading price, minus the exercise price, multiplied by the number of options,” McCormick wrote. The plaintiff came up with a value of $51 billion based on the $191.59 per-share closing price on the date of the January 2024 ruling. As previously noted, the latest Tesla price suggests the pay package could have been worth $101 billion to Musk.

Judge again rejects the Elon Musk Tesla pay plan now valued at $101 billion Read More »

The Big Nonprofits Post

Nonprofits / Blake Jones / November 29, 2024

There are lots of great charitable giving opportunities out there right now.

The first time that I served as a recommender in the Survival and Flourishing Fund (SFF) was back in 2021. I wrote in detail about my experiences then. At the time, I did not see many great opportunities, and was able to give out as much money as I found good places to do so.

How the world has changed in three years.

I recently had the opportunity to be an SFF recommender for the second time. This time I found an embarrassment of riches. Application quality was consistently higher, there were more than twice as many applications, and essentially all applicant organizations were looking to scale their operations and spending.

That means the focus of this post is different. In 2021, my primary goal was to share my perspective on the process and encourage future SFF applications. Sharing information on organizations was a secondary goal.

This time, my primary focus is on the organizations. Many people do not know good places to put donations. In particular, they do not know how to use donations to help AI go better and in particular to guard against AI existential risk. Until doing SFF this round, I did not have any great places to point them towards.

(Not all the applications were about AI. There is also a lot of attention to biological existential and catastrophic risks, some to nuclear threats, and a number of applications that took entirely different approaches.)

Organizations where I have the highest confidence in straightforward modest donations now, if your goals and model of the world align with theirs, are in bold.

A vibrant and heartwarming scene depicting various charity fundraising efforts happening in a community park. People of diverse backgrounds are engaged in activities like a bake sale with colorful desserts displayed on tables, a charity run with participants wearing numbers on their shirts, a live music performance on a small stage, and a silent auction with art pieces on display. Children are running a lemonade stand, while volunteers distribute flyers and collect donations in jars. The atmosphere is lively and festive, with balloons, banners, and smiling faces everywhere.

The SFF recommender process is highly time constrained. Even though I used well beyond the number of required hours, there was no way to do a serious investigation of all the potentially exciting applications. Substantial reliance on heuristics was inevitable.

Also your priorities, opinions, and world model could be very different from mine.

If you are considering donating a substantial amount of money, please do the level of personal research and consideration commensurate with the amount of money you want to give away.

If you are considering donating a small amount of money, or if the requirement to do personal research might mean you don’t donate to anyone at all, I caution the opposite: Only do the amount of optimization and verification and such that is worth its opportunity cost. Do not let the perfect be the enemy of the good.

For more details of how the SFF recommender process works, see my post on the process.

In addition, note that donations to some of the organizations below may not be tax deductible.

Do not let me, or anyone else, tell you any of:

What is important or what is a good cause.
What types of actions are best to make the change you want to see in the world.
What particular strategies seem promising to you.
That you have to choose according to some formula or you’re an awful person.

This is especially true when it comes to policy advocacy, and especially in AI.

If an organization is advocating for what you think is bad policy, don’t fund them!

If an organization is advocating or acting in a way you think is ineffective, don’t fund them!

Only fund people you think advance good changes in effective ways. Not cases where I think that. Cases where you think that.

Briefly on my own prioritization right now (but again you should substitute your own): I chose to deprioritize all meta-level activities and talent development, because of how much good object-level work I saw available to do, and because I expected others to often prioritize talent and meta activities. I was largely but not exclusively focused on those who in some form were helping ensure AI does not kill everyone. And I saw high value in organizations that were influencing lab or government AI policies in the right ways, and continue to value Agent Foundations style and other off-paradigm technical research approaches.

I believe that the best places to give are the places where you have local knowledge.

If you know of people doing great work or who could do great work, based on your own information, then you can fund and provide social proof for what others cannot.

The less legible to others the cause, the more excited you should be to step forward, if the cause is indeed legible to you. This keeps you grounded, helps others find the show (as Tyler Cowen says), is more likely to be counterfactual funding, and avoids information cascades or looking under streetlights for the keys.

Most importantly it avoids adverse selection. The best legible opportunities for funding, the slam dunk choices? Those are probably getting funded. The legible things that are left are the ones that others didn’t sufficiently fund yet.

If you know why others haven’t funded, because they don’t know about the opportunity?

That’s a great trade.

The process of applying for grants, raising money, and justifying your existence sucks.

A lot.

It especially sucks for many of the creatives and nerds that do a lot of the best work.

If you have to periodically go through this process, and are forced to continuously worry about making your work legible and how others will judge it, that will substantially hurt your true productivity. At best it is a constant distraction. By default, it is a severe warping effect. A version of this phenomenon is doing huge damage to academic science.

As I noted in my AI updates, the reason this blog exists is that I received generous, essentially unconditional, anonymous support to ‘be a public intellectual’ and otherwise pursue whatever I think is best. My benefactors offer their opinions when we talk because I value their opinions, but they never try to influence my decisions, and I feel zero pressure to make my work legible in order to secure future funding.

As for funding my non-Balsa work further, I am totally fine for money, but I could definitely find ways to put a larger budget to work, and shows of support are excellent for morale.

If you have money to give, and you know individuals who should clearly be left to do whatever they think is best without worrying about raising money, then giving them unconditional grants is a great use of funds, including giving them ‘don’t worry about reasonable expenses’ levels of funding.

This is especially true when combined with ‘retrospective funding,’ based on what they have already done.

Not as unconditionally, it’s also great to fund specific actions and projects and so on that you see not happening purely through lack of money, especially when no one is asking you for money.

Resist the temptation to think purely on the margin, asking only what one more dollar can do. The incentives get perverse quickly, as organizations are rewarded for putting their highest impact activities in peril. Organizations that can ‘run lean’ or protect their core activities get punished.

Also, you want to do some amount of retrospective funding. If people have done exceptional work in the past, you should be willing to give them a bunch more rope in the future, above and beyond the expected value of their new project.

Don’t make everyone constantly reprove their cost effectiveness each year, or at least give them a break. If someone has earned your trust, then if this is the project they want to do next, presume they did so because of reasons, although you are free to disagree with those reasons.

Time to talk about the organizations themselves.

Rather than offer precise rankings, I divided by cause category and into three confidence levels.

High confidence means I have enough information to be confident the organization is at least a good pick.
Medium or low confidence means exactly that – I have less confidence that the choice is wise, and you should give more consideration to doing your own research.

Low confidence is still high praise, and very much a positive assessment! The majority of SFF applicants did not make the cut, and they had already undergone selection to get that far.

If an organization is not listed, that does not mean I think they would be a bad pick – they could have asked not to be included, or I could be unaware of them or their value. I know how Bayesian evidence works, but this post is not intended as a knock on anyone, in any way. Some organizations that are not here would doubtless have been included, if I’d had more time.

I try to give a sense of how much detailed investigation and verification I was able to complete, and what parts I have confidence in versus not. Again, my lack of confidence will often be purely about my lack of time to get that confidence.

Indeed, unless I already knew them from elsewhere, assume no organizations here got as much attention as they deserve before you decide on what for you is a large donation.

I’m tiering based on how I think about donations from you, from outside SFF.

I think the regranting organizations were clearly wrong choices from within SFF, but are reasonable picks if you don’t want to do extensive research, especially if you are giving small.

In terms of funding levels needed, I will similarly divide into three categories.

They roughly mean this, to the best of my knowledge:

Low: Could likely be fully funded with less than ~$250k.

Medium: Could plausibly be fully funded with between ~$250k and ~$2 million.

High: Could probably make good use of more than ~$2 million.

These numbers may be obsolete by the time you read this. If you’re giving a large amount relative to what they might need, you might want to check with the organization first.

A lot of organizations are scaling up rapidly, looking to spend far more money than they have in the past. Everyone seems eager to double their headcount. But I’m not putting people into the High category unless I am confident they can scalably absorb more funding (although some may have now already raised that funding, so again check on that to be sure).

The person who I list as the leader of an organization will sometimes accidentally be whoever was in charge of fundraising rather than strictly the leader. Partly the reason for listing it is to give context and some of you can go ‘oh right, I know who that is,’ and the other reason is that all organization names are often highly confusing – adding the name of the organization’s leader allows you a safety check, to confirm that you are indeed pondering the same organization I am thinking of!

This is my post, so I get to list Balsa Research first. (I make the rules here.)

If that’s not what you’re interested in, you can of course skip the section.

Focus: Groundwork starting with studies to allow repeal of the Jones Act

Leader: Zvi Mowshowitz

Funding Needed: Low

Confidence Level: High

Our first target will be the Jones Act. We’re commissioning studies on its true costs, and the plan is to do more of them, and also do things like draft model repeals and explore ways to assemble a coalition and to sell and spread the results, to enable us to have a chance at repeal. Other planned cause areas include NEPA reform and federal housing policy (to build more housing where people want to live). We have one full time worker on the case.

I don’t intend to have it work on AI or assist with my other work, or to take personal compensation, unless I get donations that are dedicated to those purposes. The current fundraiser is to pay for academic studies on the Jones Act, full stop.

The pitch for Balsa, and the reason I am doing it, is in two parts.

I believe Jones Act repeal and many other abundance agenda items are neglected, tractable and important. That the basic work that needs doing is not being done, it would be remarkably cheap to do a lot of it and do it well, and that this would give us a real if unlikely chance to get a huge win if circumstances break right.

I also believe that if people do not have hope for the future, do not have something to protect and fight for, or do not think good outcomes are possible, that people won’t care about protecting the future. And that would be very bad, because we are going to need to fight to protect our future if we want to have one, or have a good one.

You got to give them hope.

I could go on, but I’ll stop there.

Focus: Zvi Mowshowitz writes a lot of words, really quite a lot.

Leader: Zvi Mowshowitz

Funding Needed: Strictly speaking none, but it all helps

Confidence Level: High

You can also of course always donate directly to my favorite charity.

By which I mean me. I always appreciate your support, however large or small.

Thanks to generous anonymous donors, I am able to write full time and mostly not worry about money. That is what makes this blog possible. I want to as always be 100% clear: I am totally, completely fine as is, as is the blog.

Please feel zero pressure here, as noted throughout there are many excellent donation opportunities out there.

Additional funds are still welcome. There are levels of funding beyond not worrying. Such additional support is always highly motivating, and also there are absolutely additional things I could throw money at to improve the blog, potentially including hiring various forms of help or even expanding to more of a full news operation or startup.

The easiest way to help (of course) is a Substack subscription or Patreon. If you want to go large then reach out to me.

As a broad category, these are organizations trying to figure things out regarding AI existential risk, without centrally attempting to either do technical work or directly to influence policy and discourse.

Focus: AI forecasting research projects, governance research projects, and policy engagement, in that order.

Leader: Daniel Kokotajlo, with Eli Lifland

Funding Needed: Medium

Confidence Level: High

Of all the ‘shut up and take my money’ applications, even before I got to participate in their tabletop wargame exercise, I judged this the most ‘shut up and take my money’-ist. At The Curve, I got to participate in the exercise and participate in discussions around it, and I’m now even more confident this is an excellent pick.

I like it going forward, and it is a super strong case for retroactive funding as well. Daniel walked away from OpenAI, and what looked to be most of his net worth, to preserve his right to speak up.

That led to us finally allowing others at OpenAI to speak up as well. This is how he wants to speak up, and try to influence what is to come, based on what he knows. I don’t know if it would have been my move, but the move makes a lot of sense. We need to back his play.

Focus: Rationality community infrastructure, LessWrong, AF and Lighthaven.

Leaders: Oliver Habryka, Raymond Arnold, Ben Pace

Funding Needed: High

Confidence Level: High

Disclaimer: I am on the CFAR board, and my writing appears on LessWrong and I have long time relationships with everyone involved, and have been to several great workshops or conferences at their campus at Lighthaven, so I was conflicted here.

I think they are doing great work and are worthy of support. There is a large force multiplier here (although that is true of a number of other organizations I list as well).

Certainly I think that if LessWrong, the Alignment Forum or the venue Lighthaven were unable to continue, especially LessWrong, that would be a major, quite bad unforced error, and I am excited by their proposed additional projects. Certainly the marginal costs here, while large (~$3 million per year), seem worthwhile to me, and far less than the fixed costs already paid.

Lightcone had been in a tricky spot for a while, because it got sued by FTX, and that made it very difficult to fundraise until it was settled, and also the settlement cost a lot of money, and OpenPhil is unwilling to fund Lightcone despite its recommenders finding Lightcone highly effective.

Now that the settlement is done, fundraising has to resume and the coffers need to be rebuilt.

Focus: AI governance, advisory and research, finding how to change decision points

Leader: Ian David Moss

Funding Needed: Medium

Confidence Level: High

Can they indeed identify ways to target key decision points, and make a big difference? One can look at their track record. I’ve been asked to keep details confidential, but based on my assessment of private information, I confirmed they’ve scored some big wins and will plausibly continue to be able to have high leverage and punch above their funding weight. You can read about some of the stuff that they can talk about here in a Founders Pledge write up. It seems important that they be able to continue their work.

Focus: Polls about AI

Leader: Daniel Colson

Funding Needed: Medium

Confidence Level: High

All those polls about how the public thinks about AI, including SB 1047? These are the people that did that. Without them, no one would be asking those questions. Ensuring that someone is asking is super helpful. With some earlier polls I was a bit worried that the wording was slanted, and that will always be a concern with a motivated pollster, but I think recent polls have been much better at this, and been reasonably close to neutral.

There are those who correctly point out that the public’s opinions are weakly held and low salience for now, and that all you’re often picking up is ‘the public does not like AI and it likes regulation.’ Fair enough, but someone still has to show this, and show it applies here, and put a lie to people claiming the public goes the other way.

(Link goes to EURAIO, this is specifically about Psychosecurity Ethics)

Focus: Summits to discuss AI respecting civil liberties and not using psychological manipulation or eroding autonomy.

Leader: Neil Watson

Funding Needed: None Right Now

Confidence Level: High

Not everything needs to be focused on purely existential risk, and even though they don’t need funding right now they probably will in the future, so I wanted to mention Psychosecurity Ethics anyway. Plenty of other things can go wrong too, and few people are thinking about many of the potential failure modes. I was excited to help this get funded, as it seems like a super cheap, excellent way to put more focus on these questions, and provides something here for those skeptical of existential concerns.

Focus: AI capabilities demonstrations to inform decision makers

Leader: Jeffrey Ladish

Funding Needed: Medium

Confidence Level: High

This is clearly an understudied approach. People need concrete demonstrations. Every time I get to talking with people in national security or otherwise get closer to decision makers who aren’t deeply into AI and in particular into AI safety concerns, you need to be as concrete and specific as possible – that’s why I wrote Danger, AI Scientist, Danger the way I did. We keep getting rather on-the-nose fire alarms, but it would be better if we could get demonstrations even more on the nose, and get them sooner, and in a more accessible way. I have confidence that Jeffrey is a good person to put this plan into action.

To donate, email [email protected].

Focus: Making YouTube videos about AI safety, starring Rob Miles

Leader: Rob Miles

Funding Needed: Low

Confidence Level: High

I think these are pretty great videos in general, and given what it costs to produce them we should absolutely be buying their production. If there is a catch, it is that I am very much not the target audience, so you should not rely too much on my judgment of what is and isn’t effective video communication on this front, and you should confirm you like the cost per view.

To donate, join his patreon or contact him directly.

Focus: Facilitation of the AI scenario planning game Intelligence Rising.

Leader: Caroline Jeanmaire

Funding Needed: Low

Confidence Level: High

I haven’t had the opportunity to play Intelligence Rising, but I have read the rules to it, and heard a number of excellent after action reports (AARs), and played Daniel Kokotajlo’s version. The game is clearly solid, and it would be good if they continue to offer this experience and if more decision makers play it.

To donate, reach out to [email protected].

Focus: A series of sociotechnical reports on key AI scenarios, governance recommendations and conducting AI awareness efforts.

Leader: David Kristoffersson

Funding Needed: Medium (for funding their Scenario Planning only)

Confidence Level: Medium

They have three tracks.

I am not so interested in their Governance Research and AI Awareness tracks, where I believe there are many others, some of which seem like better bets.

Their Scenario Planning track is more exciting. It is not clear who else is doing this work, and having concrete scenarios to consider and point to, and differentiate between, seems highly valuable. If that interests you, I would check out their reports in this area, and see if you think they’re doing good work.

Their donation page is here.

Focus: Conferences and advice on x-risk for those giving >$1 million per year

Leader: Simran Dhaliwal

Funding Needed: Medium

Confidence Level: Low

They also do some amount of direct grantmaking, but are currently seeking funds for their conferences. They involve top experts including Hinton and Bengio and by several accounts put on strong conferences. The obvious question is why, given all those giving so much, why this isn’t able to self-fund, and I am always nervous about giving money to those who focus on getting others to in turn give more money, as I discussed last time. I presume this does successfully act as a donation multiplier, if you are more comfortable than I am with that sort of strategy.

To inquire about donating, submit a query using the contact form at the bottom of their website.

Some of these organizations also look at bio policy or other factors, but I judge those here as being primarily concerned with AI.

In this area, I am especially keen to rely on people with good track records, who have shown that they can build and use connections and cause real movement. It’s so hard to tell what is and isn’t effective, otherwise. Often small groups can pack a big punch, if they know where to go, or big ones can be largely wasted – I think that most think tanks on most topics are mostly wasted even if you believe in their cause.

Focus: AI research, field building and advocacy

Leaders: Dan Hendrycks

Funding Needed: High

Confidence Level: High

They played a key role in SB 1047 getting this far, they did the CAIS Statement on AI Risk, and in many other ways they’ve clearly been punching well above their weight in the advocacy space. The other arms are no slouch either, lots of great work here.

If you want to focus on their policy, then you can fund their 501c(4), the Action Fund, since 501c(3)s are limited in how much they can spend on political activities, keeping in mind the tax implications of that..

It would be pretty crazy if we didn’t give them the funding they need.

Focus: At this point, primarily AI policy advocacy, plus some research

Leaders: Malo Bourgon, Eliezer Yudkowsky

Funding Needed: High

Confidence Level: High

MIRI, concluding that it is highly unlikely alignment will make progress rapidly enough otherwise, has shifted its strategy to largely advocate for major governments coming up with an international agreement to halt AI progress and to do communications, although research still looks to be a large portion of the budget, and they have dissolved its agent foundations team. That is not a good sign for the world, but it does reflect their beliefs.

They have accomplished a lot, and I strongly believe they should be funded to continue to fight for a better future however they think is best, even when I disagree with their approach.

This is very much a case of ‘do this if and only if this aligns with your model and preferences.’

Focus: Tech policy research, thought leadership, educational outreach to government

Leader: Grace Meyer

Funding Needed: Medium

Confidence Level: High

FAI is centrally about innovation. I am all for innovation in most situations as well. Innovation is good, actually, as is building things and letting people do things. But in AI people calling for ‘supporting innovation’ are often using that as an argument against all regulation of AI, and indeed I am dismayed to see so many push so hard on this exactly in the one place I think they are deeply wrong – we could work together on it almost anywhere else.

Indeed, their Chief Economist and resident AI studier Samuel Hammond, who launched their AI safety advocacy efforts in April 2023, initially opposed SB 1047, after revisions moving to what I interpret as a neutral position, and I famously had some strong disagreements with his 95 theses on AI although I agreed far more than I disagreed, and have many disagreements with AI and Leviathan as well.

Yet here they are rather high on the list. I have strong reasons to believe that we are closely aligned on key issues including compute governance, and private reasons to believe that FAI has been effective and we can expect that to continue, and its other initiatives also seem good. We don’t have to agree on everything else, so long as we all want good things and are trying to figure things out, and I’m confident that is the case here.

I am especially excited that they can speak to the Republican side of the aisle in the R’s native language, which is difficult for most in this space to do.

An obvious caveat is that if you are not interested in the non-AI pro-innovation part of the agenda (I certainly approve, but it’s not obviously a high funding priority for most readers) then you’ll want to ensure it goes where you want it.

Focus: Lobbying Congress to adapt mandatory AI safety standards

Leader: Jason Green-Lowe

Funding Needed: Medium

Confidence Level: High

They’re a small organization starting out. Their biggest action so far has been creating a model AI governance bill, which I reviewed in depth. Other than too-low compute thresholds throughout, their proposal was essentially ‘the bill people are hallucinating when they talk about SB 1047, except very well written.’ I concluded it was a very thoughtful model bill, written to try and do a specific thing. Most of its choices made a lot of sense, and it is important work to have a bill like that already drafted and ready to go. There are a lot of futures where we don’t get a bill until some catastrophic event or other catastrophe, and then we suddenly pass something in a hurry.

Focus: Youth activism on AI safety issues

Leader: Sneha Revanur

Funding Needed: Medium

Confidence Level: High

They have done quite a lot on a shoestring budget by using volunteers, helping with SB 1047 and in several other places. Now they are looking to turn pro, and would like to not be on a shoestring. I think they have clearly earned that right. The caveat is risk of ideological capture. Youth organizations tend to turn to left wing causes.

The risk here is that this effectively turns mostly to AI ethics concerns. It’s great that they’re coming at this without having gone through the standard existential risk ecosystem, but that also heightens the ideological risk. I think it’s still worth it.

To donate, go here.

Focus: AI governance standards and policy.

Leader: Caroline Jeanmaire

Funding Needed: Medium

Confidence Level: High

I’ve seen credible sources saying they do good work, and that they substantially helped orient the EU AI Act to at least care at all about frontier general AI. The EU AI Act was not a good bill, but it could easily have been a far worse one, doing much to hurt AI development while providing almost nothing useful for safety. We should do our best to get some positive benefits out of the whole thing.

They’re also active around the world, including the USA and China.

Focus: Specifications for good AI safety, also directly impacting EU AI policy

Leader: Simeon Campos

Funding Needed: Medium

Confidence Level: High

I’ve known Simeon for a while. I am impressed. He knows his stuff, he speaks truth to power. He got good leverage during the EU AI Act negotiations, does a bunch of good invisible background stuff, and in this case I am in position to know about some of it. I definitely want to help him cook.

To donate, go here.

Focus: Papers and projects for ‘serious’ government circles, meetings with same.

Leader: Peter Wildeford

Funding Needed: High

Confidence Level: Medium

I have a lot of respect for Peter Wildeford, and they’ve clearly put in good work and have solid connections down, including on the Republican side where better coverage is badly needed. My verification level on degree of impact here (past and projected) is less definite here than with some of the High-level similar orgs, but they are clearly doing the thing, and this clearly crosses the ‘should be funded’ line in a sane world.

To donate, go here.

Focus: Accelerating the writing of AI safety standards

Leaders: Ariel Gil and Jonathan Happel

Funding Needed: Medium

Confidence Level: Medium

They help facilitate the writing of AI safety standards, for EU/UK/USA. They have successfully gotten some of their work officially incorporated, and another recommender with a standards background was impressed by the work and team. This is one of the many things that someone has to do, and where if you step up and do it and no one else does that can go pretty great. Having now been involved in bill minutia myself, I know it is thankless work, and that it can really matter.

To donate, reach out to [email protected].

Focus: International AI safety conferences

Leader: Fynn Heide and Conor McGurk

Funding Needed: Medium

Confidence Level: Medium

They run the IDAIS series of conferences, including successful ones involving China. I do wish I had a better model of what makes such a conference actually matter versus not mattering, but these sure seem like they should matter, and certainly well worth their costs to run them.

Focus: UK Policy Think Tank focusing on ‘extreme AI risk and biorisk policy.’

Leader: Angus Mercer

Funding Needed: High

Confidence Level: Medium

The UK has shown promise in its willingness to shift its AI regulatory focus to frontier models in particular. It is hard to know how much of that shift to attribute to any particular source, or otherwise measure how much impact there has been or might be on final policy.

They have endorsements of their influence from among others Toby Ord, Former Special Adviser to the UK Prime Minister Logan Graham and Senior Policy Adviser Nitarshan Rajkumar.

I reached out to a source with experience in the UK government who I trust, and they reported back they are a fan and pointed to some good things they’ve helped with. There was a general consensus that they do good work, and those who investigated where impressed.

The concern is that their funding needs are high, and they are competing against many others in the policy space, many of which have very strong cases. But they seem clearly like a solid choice.

To donate, go here.

Focus: Advocating for a pause on AI, including via in-person protests

Leader: Holly Elmore

Funding Level: Medium

Confidence Level: Medium

Some people say that those who believe we should pause AI would be better off staying quiet about it, rather than making everyone look foolish. Even though I very much do not think outright pausing AI is anything close to our first best policy at the moment, I think that those who believe we should pause AI should stand up and say we should pause AI. I very much appreciate people standing up, entering the arena and saying what they believe in, including quite often in my comments. Let the others mock all they want.

If you agree with Pause AI that the right move is to pause AI, then you should likely be excited to fund this. If you disagree, you have better options. But I’m happy that they are going for it.

Focus: Get the word out and also organize conferences

Leader: Otto Barten

Funding Needed: Low

Confidence Level: Medium

Mostly this is the personal efforts of Otto Barten, ultimately advocating for a conditional pause. For modest amounts of money, he’s managed to have a hand in some high profile existential risk events and get the first x-risk related post into TIME magazine. It seems worthwhile to pay the modest amount to ensure he can keep doing what he is doing, in the way he thinks is best.

Focus: Foundations and demand for international cooperation on AI governance and differential tech development

Leader: Konrad Seifert and Maxime Stauffer

Funding Needed: Medium

Confidence Level: Medium

As with all things diplomacy, hard to tell the difference between a lot of talk and things that are actually useful. Things often look the same either way for a long time. A lot of their focus is on the UN, so update either way based on how useful you think that approach is. They are doing a lot of attempted Global South coordination on this.

Focus: Legal team for lawsuits on catastrophic risk and to defend whistleblowers.

Leader: Tyler Whitmer

Funding Needed: Medium

Confidence Level: Low

I wasn’t sure where to put them, but I suppose lawsuits are kind of policy by other means in this context, or close enough? I buy the core idea, which is that having a legal team on standby for catastrophic risk related legal action in case things get real quickly is a good idea, and I haven’t heard anyone else propose this, although I do not feel qualified to vet the operation.

While they are open to accepting donations, they’re not yet set up to take a ton of smaller donations (yet). Donors who are interested in making relatively substantial donations or grants should contact [email protected].

This category should be self-explanatory. Unfortunately, a lot of good alignment work still requires charitable funding. The good news is that there is a lot more funding, and willingness to fund, than there used to be, and also the projects generally look more promising.

The great thing about interpretability is that you can be confident you are dealing with something real. The not as great thing is that this can draw too much attention to interpretability, and that you can fool yourself into thinking that All You Need is Interpretability.

The good news is that several solid places can clearly take large checks.

I didn’t investigate too deeply on top of my existing knowledge here, because at SFF I had limited funds and decided that direct research support wasn’t a high enough priority, partly due to it being sufficiently legible. We should be able to find money on the sidelines eager to take these opportunities.

Formerly ARC Evaluations.

Focus: Model evaluations

Leaders: Beth Barnes, Emma Abele, Chris Painter, Kit Harris

Funding Needed: None Whatsoever

Confidence Level: High

Originally I wrote that we hoped to be able to get large funding for METR via non-traditional sources. That has happened – METR got major funding recently. That’s great news. It also means there is no plausible ‘funding gap’ here for now.

If it ever does need funding again, METR has proven to be the gold standard for outside evaluations of potentially dangerous frontier model capabilities. We very much need these outside evaluations, and to give the labs every reason to use them and no excuse not to use them. In an ideal world the labs would be fully funding METR, but they’re not. So this becomes a place where we can confidently invest quite a bit of capital, make a legible case for why it is a good idea, and know it will probably be well spent.

Focus: Theoretically motivated alignment work

Leader: Jacob Hinton

Funding Needed: High

Confidence Level: High

There’s a long track record of good work here, and Paul Christiano remains excited. If you are looking to fund straight up alignment work and don’t have a particular person or small group in mind, this is certainly a safe bet to put additional funds to good use and attract good talent.

Focus: Evaluations, especially versus deception, some interpretability and governance.

Leader: Marius Hobbhahn

Funding Needed: High

Confidence Level: High

This is an excellent thing to focus on, and one of the places we are most likely to be able to show ‘fire alarms’ that make people sit up and notice. Their first year seems to have gone well, one example would be their presentation at the UK safety summit that LLMs can strategically deceive their primary users when put under pressure. They will need serious funding to fully do the job in front of them, hopefully like METR they can be helped by the task being highly legible.

To donate, reach out to [email protected].

Focus: Allow Roman Yampolskiy to continue his research and pursue a PhD

Leader: Roman Yampolskiy

Funding Needed: Low

Confidence Level: High

If this still hasn’t happened by the time you read this, and there is danger he won’t be able to do the PhD, then obviously someone should fix that. His podcast on Lex Fridman was a great way to widen the audience, and it is clear he says what he believes and is pursuing what he thinks might actually help. He is the doomiest of doomers, and I’m glad he is not holding back on that, even if I disagree on the assessment and think it’s not the ideal look. Because the ideal is to say what you think.

Focus: Interpretability research

Leader: Jesse Hoogland

Funding Needed: Medium

Confidence Level: High

Timaeus focuses on interpretability work and sharing their results. The set of advisors is excellent, including Davidad and Evan Hubinger. Evan, John Wentworth and Vanessa Kosoy have offered high praise, and there is evidence they have impacted top lab research agendas. They’re done what I think is solid work, although I am not so great at evaluating papers directly. If you’re interested in directly funding interpretability research, that all makes this seem like a slam dunk.

To donate, get in touch with Jesse at [email protected]. If this is the sort of work that you’re interested in doing, they also have a discord at http://devinterp.com/discord.

Focus: Mechanistic interpretability of how inference breaks down

Leader: Paul Riechers and Adam Shai

Funding Needed: Medium

Confidence Level: High

I am not as high on them as I am on Timaeus, but they have given reliable indicators that they will do good interpretability work. I’d feel comfortable backing them.

Focus: Interpretability and other alignment research, incubator, hits based approach

Leader: Adam Gleave

Funding Needed: High

Confidence Level: Medium

Hits based is the right approach to research. I’ve gotten confirmation that they’re doing the real thing here. In an ideal world everyone doing the real thing would get supported. But my verification is secondhand.

Focus: AI alignment research on hierarchical agents and multi-system interactions

Leader: Jan Kulveit

Funding Needed: Medium

Confidence Level: Medium

I like the focus here on agents and their interactions, and from what I saw I think he is generally thinking well. If one wants to investigate further, he has an AXRP podcast episode, which I haven’t listened to.

To donate, reach out to [email protected], and note that you are interested in donating to ACS specifically.

Focus: AI safety hackathons

Leaders: Esben Kran, Jason Schreiber

Funding Needed: Medium

Confidence Level: Low

I’m confident in their execution of the idea. My doubt here is on the level of ‘is AI safety something that benefits from hackathons.’ Is this something one can, as it were, hack together usefully? Are the hackathons doing good counterfactual work? Or is this a way to flood the zone with more variations on the same ideas? As with many orgs on the list, this one makes sense if and only if you buy the business model.

Focus: Interpretability, tools for AI control, and so forth. New org.

Leaders: Jacob Steinhardt, Sarah Schwettmann

Funding Needed: High

Confidence Level: Low

This would be a new organization. I have confirmation the team is credible. The plan is highly ambitious, with planned scale well beyond what SFF could have funded. I haven’t done anything like the investigation into their plans and capabilities you would need before placing a bet that big, as AI research of all kinds gets expensive quickly. If there is sufficient appetite to scale the amount of privately funded direct work of this type, then this seems like a fine place to look.

To donate, reach out to [email protected].

Focus: Guaranteed safe AI

Leaders: Evan Miyazono

Funding Needed: Medium

Confidence Level: Low

My hesitancy here is my hesitancy regarding the technical approach. I still can’t see how the guaranteed safe AI plan can work. I’m all for trying it, it is clearly something people should try given how many very smart people find promise in it. I sure hope I’m wrong and the approach is viable. If you find it promising, this looks much better.

They receive donations from here, or you can email them at [email protected].

Right now it looks likely that AGI will be based around large language models (LLMs). That doesn’t mean this is inevitable. I would like our chances better if we could base our ultimate AIs around a different architecture, one that was more compatible with being able to get it to do what we would like it to do.

One path for this is agent foundations, which involves solving math to make the programs work instead of relying on inscrutable giant matrices.

Even if we do not manage that, decision theory and game theory are potentially important for navigating the critical period in front of us, for life in general, and for figuring out what the post-transformation AI world might look like, and thus what choice we make now might do to impact that.

There are not that many people working on these problems. Actual Progress would be super valuable. So even if we expect the median outcome does not involve enough progress to matter, I think it’s still worth taking a shot.

The flip side is you worry about people ‘doing decision theory into the void’ where no one reads their papers or changes their actions. That’s a real issue. As is the increased urgency of other options. Still, I think these efforts are worth supporting, in general.

Focus: AI alignment via agent foundations

Leaders: Tamsin Leake

Funding Needed: Medium

Confidence Level: High

I have funded Orthogonal in the past. They are definitely doing the kind of work that, if it succeeded, might actually amount to something, and would help us get through this to a future world we care about. It’s a long shot, but a long shot worth trying. My sources are not as enthusiastic as they once were, but there are only a handful of groups trying that have any chance at all, and this still seems like one of them.

Focus: Math for AI alignment

Leaders: Brendan Fong and David Spivak.

Funding Needed: Medium

Confidence Level: High

Topos is essentially Doing Math to try and figure out what to do about AI and AI Alignment. I’m very confident that they are qualified to (and actually will) turn donated money (partly via coffee) into math, in ways that might help a lot. I am also confident that the world should allow them to attempt this.

Ultimately it all likely amounts to nothing, but the upside potential is high and the downside seems very low. I’ve helped fund them in the past and am happy about that.

To donate, go here.

Focus: Two people doing research at MIRI, in particular Sam Eisenstat

Leader: Sam Eisenstat

Funding Needed: Medium

Confidence Level: High

Given Sam Eisenstat’s previous work it seems worth continuing to support him, including an additional researcher of his choice. I still believe in this stuff being worth working on, obviously only support if you do as well.

To donate, contact [email protected].

Focus: This research agenda, with this status update, examining intelligence

Leader: Vanessa Kosoy

Funding Needed: Medium

Confidence Level: High

This is Vanessa Kosoy and Alex Appel, who have another research agenda formerly funded by MIRI that now needs to stand on its own after their refocus. I once again believe this work to be worth continuing even if the progress isn’t what one might hope. I wish I had the kind of time it takes to actually dive into these sorts of theoretical questions, but alas I do not, or at least I’ve made a triage decision not to.

(Link goes to a Google doc with more information, no website yet.)

Focus: Searching for a mathematical basis for metaethics.

Leader: Alex Zhu

Funding Needed: Small

Confidence Level: Low

Alex Zhu has run iterations of the Math & Metaphysics Symposia, which had some excellent people in attendance, and intends partly to do more things of that nature. He thinks eastern philosophy contains much wisdom relevant to developing a future ‘decision-theoretic basis of metaethics’ and plans on an 8+ year project to do that.

I’ve seen plenty of signs that the whole thing is rather bonkers, but also strong endorsements from a bunch of people I trust that there is good stuff here, and the kind of crazy that is sometimes crazy enough to work. So there’s a lot of upside. If you think this is kind of approach has a chance of working, this could be very exciting.

To donate, message Alex at [email protected].

Focus: Game theory for cooperation by autonomous AI agents

Leader: Vincent Conitzer

Funding Needed: Medium

Confidence Level: Low

This is an area MIRI and the old rationalist crowd thought about a lot back in the day. There are a lot of ways for advanced intelligences to cooperate that are not available to humans, especially if they are capable of doing things in the class of sharing source code or can show their decisions are correlated with each other. With sufficient capability, any group of agents should be able to act as if it is a single agent, and we shouldn’t need to do the game theory for them in advance either. I think it’s good things to be considering, but one should worry that even if they do find answers it will be ‘into the void’ and not accomplish anything. Based on my technical analysis I wasn’t convinced Focal was going to sufficiently interesting places with it, but I’m not at all confident in that assessment.

To donate, reach out to Vincent directly at [email protected] to be guided through the donation process.

This section is the most fun. You get unique projects taking big swings.

Focus: Drexlerian Nanotechnology

Leaders: Eric Drexler, of course

Funding Needed; Medium

Confidence Level: High

Yes, it’s Eric Drexler looking for funding for better understanding nanotechnology, including by illustrating it via games. This seems like a clear case of ‘shut up and take my money.’ The catch is that he wants to open source the tech, and there are some obvious reasons why open sourcing nanotechnology might not be a wonderful idea? This is another case of it being fine for now, and perhaps there being a time in the future when it would need to stop, which should be obvious. Given that it should be obvious and how brilliant Drexler is and how much we need to get lucky somewhere, I’m very willing to gamble.

To donate, reach out to [email protected].

Focus: Feeding people with resilient foods after a potential nuclear war

Leaders: David Denkenberger

Funding Needed: High

Confidence Level: Medium

As far as I know, no one else is doing the work ALLFED is doing. A resilient food supply ready to go in the wake of a nuclear war could be everything. There’s a small but real chance that the impact is enormous. In my 2021 SFF round, I went back and forth with them several times over various issues, ultimately funding them, you can read about those details here.

I think all of the concerns and unknowns from last time essentially still hold, as does the upside case. I decided I wasn’t going to learn more without a major time investment, and that I didn’t have the ability to do that investment.

If you are convinced by the viability of the tech and ability to execute, then there’s a strong case that this is a very good use of funds, especially if you are an ‘AI skeptic’ and also if your model of AI political dynamics includes a large chance of nuclear war.

Research and investigation on the technical details seems valuable here. If we do have a viable path to alternative foods and don’t fund it, that’s a pretty large miss, and I find it highly plausible that this could be super doable and yet not otherwise done.

Focus: Collaborations for tools to increase civilizational robustness to catastrophes

Leader: Colby Thompson

Funding Needed: High

Confident Level: High

The principle of ‘a little preparation now can make a huge difference to resilience and robustness in a disaster later, so it’s worth doing even if the disaster is not so likely’ generalizes. Thus, the Good Ancestor Foundation, targeting nuclear war, solar flares, internet and cyber outages, and some AI scenarios and safety work.

A particular focus is archiving data and tools, enhancing synchronization systems and designing a novel emergency satellite system (first one goes up in June) to help with coordination in the face of disasters. They’re also coordinating on hardening critical infrastructure and addressing geopolitical and human rights concerns. They’ve also given out millions in regrants.

One way I know they make good decisions is they help facilitate the funding for my work. They have my sincerest thanks. Which also means there is a conflict of interest, so take that into account.

To donate, click here.

Focus: Building charter cities

Leader: Kurtis Lockhart

Funding Needed: Medium

Confidence Level: Medium

I do love charter cities. There is little question they are attempting to do a very good thing and are sincerely going to attempt to build a charter city in Africa, where such things are badly needed. Very much another case of it being great that someone is attempting to do this. Seems like a great place for people who don’t think transformational AI is on its way but do understand the value here.

Focus (of this proposal was): Creating primates from cultured edited stem cells

Leaders: Sergiy Velychko and Rudiger Behr

Funding Needed: High

Confidence Level: Low

The Primate Center is much bigger than any one project, but this project was intriguing – if you are donating because of this project, you’ll want to make sure the money goes for this specific project. The theory says that you should be able to create an embryo directly from stem cells, including from any combination of genders, with the possibility of editing their genes. If it worked, this could be used for infertility, allowing any couple to have a child, and potentially the selection involved could be used for everything from improved health to intelligence enhancement. The proposed project in particular is to do this in primates.

I can’t speak directly to verify the science, and there are those who think any existential risk considerations probably arrive too late to matter, and of course you will want to consider any potential ethical concerns, but if you see substantial chance it works and think of this purely as the ultimate infertility treatment, that is already amazing value.

Focus: Whole brain emulation

Leader: Randal Koene

Funding Needed: Medium

Confidence Level: Low

At this point, if it worked in time to matter, I would be willing to roll the dice on emulations. What I don’t have is much belief that it will work, or the time to do a detailed investigation into the science. So flagging here, because if you look into the science and you think there is a decent chance, this becomes a good thing to fund.

Focus: Scanning DNA synthesis for potential hazards

Leader: Kevin Esvelt, Andrew Yao and Raphael Egger

Funding Needed: Medium

Confidence Level: Medium

It is certainly an excellent idea. Give everyone fast, free, cryptographically screening of potential DNA synthesis to ensure no one is trying to create something we do not want anyone to create. AI only makes this concern more urgent. I didn’t have time to investigate and confirm this is the real deal as I had other priorities even if it was, but certainly someone should be doing this.

There is also another related effort, Secure Bio, if you want to go all out. I would fund Secure DNA first.

To donate, contact them here.

Focus: Increasing capability to respond to future pandemics, Next-gen PPE, Far-UVC.

Leader: Jake Swett

Funding Needed: Medium

Confidence Level: Medium

There is no question we should be spending vastly more on pandemic preparedness, including far more on developing and stockpiling superior PPE and in Far-UVC. It is rather a shameful that we are not doing that, and Blueprint Biosecurity plausibly can move substantial additional investment there. I’m definitely all for that.

To donate, reach out to [email protected] or head to the Blueprint Bio PayPal Giving Fund.

Focus: AI enabled biorisks, among other things.

Leader: Patrick Stadler

Funding Needed: Low

Confidence Level: Low

Everything individually looks worthwhile but also rather scattershot. Then again, who am I to complain about a campaign for e.g. improved air quality? My worry is still that this is a small operation trying to do far too much, some of it that I wouldn’t rank too high as a priority, and it needs more focus, on top of not having that clear big win yet.

Donation details are at the very bottom of this page.

There were lots of great opportunities in SFF this round. I was going to have an embarrassment of riches I was excited to fund.

Thus I decided quickly that I would not be funding any regrating organizations. If you were in the business of taking in money and then shipping it out to worthy causes, well, I could ship directly to highly worthy causes, so there was no need to have someone else do the job again, or reason to expect them to do better.

That does not mean that others should not consider such donations.

I see two important advantages to this path.

Regranters can offer smaller grants that are well-targeted.
Regranters save you a lot of time.

Thus, if you are making a ‘low effort’ donation, and think others you trust that share your values to invest more effort, it makes more sense to consider regranters.

Focus: Give out grants based on recommenders, primarily to 501c(3) organizations

Leaders: Andrew Critch and Jaan Tallinn

Funding Needed: High

Confidence Level: High

If I had to choose a regranter right now to get a large amount of funding, my pick would be to give it to the SFF process. The applicants and recommenders are already putting in their effort, and it is very clear there are plenty of exciting places to put additional funds. The downside is that SFF can’t ‘go small’ efficiently on either end, so it isn’t good at getting small amounts of funding to individuals. If you’re looking to do that in particular, and can’t do it directly, you’ll need to look at other options.

Due to their operational scale, SFF is best suited only for larger donations.

Focus: Regranters to AI safety, existential risk, EA meta projects, creative mechanisms

Leader: Austin Chen (austin at manifund.org).

Funding Needed: Medium

Confidence Level: Medium

This is a regranter that gives its money to its own regranters, one of which was me, for unrestricted grants. They’re the charity donation offshoot of Manifold. They’ve played with crowdfunding, and with impact certificates, and ACX grants. They help run Manifest.

You’re essentially hiring these people to keep building a website and trying alternative funding allocation mechanisms, and for them to trust the judgment of selected regranters. That seems like a reasonable thing to do if you don’t otherwise know where to put your funds and want to fall back on a wisdom of crowds of sorts. Or, perhaps, if you actively want to fund the cool website.

Manifold itself did not apply, but I would think that would also be a good place to invest or donate in order to improve the world. It wouldn’t even be crazy to go around subsidizing various markets. If you send me manna there, I will set aside and use that manna to subsidize markets when it seems like the place to do that.

If you want to support Manifold itself, you can donate or buy a SAFE, contact Austin.

Also I’m a regranter at Manifund, so if you wanted to, you could use that to entrust me with funds to regrant. As you can see I certainly feel I have plenty of good options here if I can’t find a better local one, and if it’s a substantial amount I’m open to general directions (e.g. ensuring it happens relatively quickly, or a particular cause area as long as I think it’s net positive, or the method of action or theory of impact).

Focus: Spinoff of LTFF, grants for AI safety projects

Leader: Thomas Larsen

Funding Needed: Medium

Confidence Level: Medium

Seems very straightforwardly exactly what it is, a standard granter, usually in the low six figure range. Fellow recommenders were high on Larsen’s ability to judge projects. If you think this is better than you can do on your own and you want to fund such projects, then sure, go for it.

Focus: Grants of 4-6 figures mostly to individuals, mostly for AI existential risk

Leader: Caleb Parikh (among other fund managers)

Funding Needed: High

Confidence Level: Low

The pitch on LTFF is that it is a place for existential risk people who need modest cash infusions to ask for them, and to get them without too much overhead or distortion. Looking over the list of grants, there is at least a decent hit rate. One question is, are the marginal grants a lot less effective than the average grant?

My worry is that I don’t know the extent to which the process is accurate, fair, favors insiders or extracts a time or psychic tax on participants, or rewards ‘being in the EA ecosystem’ or especially the extent to which the net effects are distortionary and bias towards legibility and standardized efforts. Or the extent to which people use the system to extract funds without actually doing anything.

That’s not a ‘I think the situation is bad,’ it is a true ‘I do not know.’ I doubt they know either.

What do we know? They say applications should take 1-2 hours to write and between 10 minutes and 10 hours to evaluate, although that does not include time forming the plan, and this is anticipated to be an ~yearly process long term. And I don’t love that this concern is not listed under reasons not to choose to donate to the fund (although the existence of that list at all is most welcome, and the reasons to donate don’t consider the flip side either).

Focus: Regrants, fellowships and events

Leader: Allison Duettmann

Funding Needed: Medium

Confidence Level: Low

Foresight also does other things. The focus here was their AI existential risk grants, which they offer on a rolling basis. I’ve advised them on a small number of potential grants, but they haven’t asked often as of yet. The advantage on the regrant side would be to get outreach that wasn’t locked too tightly into the standard ecosystem. The other Foresight activities all seem clearly like good things, but the bar these days is high and since they weren’t the topic of the application I didn’t investigate. They’ve invited me to an event, but I haven’t been able to find time to go.

Focus: The Athena Hotel aka The EA Hotel as catered host for EAs in UK

Leader: Greg Colbourn

Funding Needed: Medium

Confidence Level: Low

I love the concept of a ‘catered hotel’ where select people can go to be supported in whatever efforts seem worthwhile. If you are looking to support a very strongly EA-branded version of that, which I admit that I am not, then here you go.

I am widely skeptical of prioritizing AI safety talent funnels at this time.

The reason is simple. If we have so many good organizations already, in need of so much funding, why do we need more talent funnels? Is talent our limiting factor? Are we actually in danger of losing important talent?

The situation was very different last time. We had more funding than I felt we had excellent places to put it. Indeed, I solicited and then gave a grant to Emergent Ventures India. That’s a great way to promote development of talent in general, the grants are very small and have large impacts, and Tyler Cowen is an excellent evaluator and encourager of and magnet for talent.

Now I look at all the organizations here, and I don’t see a shortage of good talent. If anything, I see a shortage of ability to put that talent to good use.

The exception is leadership and management. There remains, it appears, a clear shortage of leadership and management talent across all charitable space, and startup space, and probably flat out all of space.

Which means if you are considering stepping up and doing leadership and management, then that is likely more impactful than you might at first think.

If there was a strong talent funnel specifically for leadership or management, that would be a very interesting funding opportunity. And yes, of course there still need to be some talent funnels. Right now, my guess is we have enough, and marginal effort is best spent elsewhere.

But also high returns from developing good talent are common, so disagreement here is reasonable. This is especially true if people can be placed ‘outside the ecosystem’ where they won’t have to compete with all the usual suspects for their future funding. If you can place them into government, that’s even better. To the extent that is true, it makes me more excited.

Focus: Learning by doing, participants work on a concrete project in the field

Leaders: Remmelt Ellen and Linda Linsefors

Funding Needed: Low

Confidence Level: High

By all accounts they are the gold standard for this type of thing. Everyone says they are great, I am generally a fan of the format, I buy that this can punch way above its weight or cost. If I was going to back something in this section, I’d start here.

Donors can reach out to Remmelt at [email protected], or leave a donation at Manifund to help cover stipends.

Focus: Paying academics small stipends to move into AI safety work

Leaders: Peter Salib (psalib @ central.uh.edu), Yonathan Arbel (yarbel @ law.ua.edu) and Kevin Frazier (kfrazier2 @ stu.edu).

Funding Needed: Low

Confidence Level: High

This strategy is potentially super efficient. You have an academic that is mostly funded anyway, and they respond to remarkably small incentives to do something they are already curious about doing. Then maybe they keep going, again with academic funding. If you’re going to do ‘field building’ and talent funnel in a world short on funds for those people, this is doubly efficient. I like it.

To donate, message one of leaders at the emails listed above.

Focus: Fellowships for Drexlerian functional nanomachines, high-throughput tools and discovering new superconductors

Leader: Benjamin Reinhardt

Funding Needed: Medium

Confidence Level: Medium

My note to myself of ‘is it unfair to say we should first fund literal Eric Drexler?’, who is also seeking funding, is indeed a tad unfair, also illustrates how tough it is out there looking for funding. I have confirmation that Reinhardt knows his stuff, and we certainly could use more people attempting to build revolutionary hardware. If the AI is scary enough to make you not want to build the hardware, it would figure out how to build the hardware anyway, so you might as well find out now.

So if you’re looking to fund a talent funnel, this seems like a good choice.

To donate, go here.

Focus: Fellowships to other organizations, such as Future Society, Safer AI and FLI.

Leader: Cillian Crosson (same as Tarbell for now but she plans to focus on Tarbell)

Funding Needed: Medium

Confidence Level: Medium

They run two fellowship cohorts a year. They seem to place people into a variety of solid organizations, and are exploring the ability to get people into various international organizations like the OECD, UN or European Commission or EU AI Office. The more I am convinced people will actually get inside meaningful government posts, the more excited I will be.

To donate, contact [email protected].

Focus: Researcher mentorship for those new to AI safety.

Leaders: Ryan Kidd and Christian Smith.

Funding Needed: High

Confidence Level: Medium

MATS is by all accounts very good at what they do and they have good positive spillover effects on the surrounding ecosystem. If (and only if) you think that what they do, which is support would-be alignment researchers starting out, is what you want to fund, then you should absolutely fund them. That’s a question of prioritization.

Focus: X-risk residencies, workshops, coworking in Prague, fiscal sponsorships

Leader: Irena Kotikova

Funding Needed: Medium

Confidence Level: Medium

I see essentially two distinct things here.

First, you have the umbrella organization, offering fiscal sponsorship for other organizations. Based on what I know from the charity space, this is a highly valuable service – it was very annoying getting Balsa a fiscal sponsor, even though we ultimately found a very good one that did us a solid, and also annoying figuring out how to be on our own going forward.

Second, you have various projects around Prague, which seem like solid offerings in that class of action of building up EA-style x-risk actions in the area, if that is what you are looking for. So you’d be supporting some mix of those two things.

To donate, contact [email protected].

Focus: Small grants to individuals to help them develop their talent

Leader: Tyler Cowen

Funding Needed: Medium

Confidence Level: High

I’m listing this at the end of the section as a bonus entry. They are not like the other talent funnels in several important ways.

It’s not about AI Safety. You can definitely apply for an AI Safety purpose, he’s granted such applications in the past, but topics run across the board, well beyond the range otherwise described in this post.
Decisions are quick and don’t require paperwork or looking legible. Tyler Cowen makes the decision, and there’s no reason to spend much time on your end either.
There isn’t a particular cause area this is trying to advance, and he’s not trying to steer people to do any particular thing. Just to be more ambitious, and be able to get off the ground and build connections and so on. It’s not prescriptive.

I strongly believe this is an excellent way to boost the development of more talent, as long as money is serving as a limiting factor on the project, and that it is great to develop talent even if you don’t get to direct or know where it is heading. Sure, I get into rhetorical arguments with Tyler all the time, around AI and also other things, and we disagree strongly about some of the most important questions where I don’t understand how he can continue to have the views he does, but this here is still a great project, an amazingly cost-efficient intervention.

Focus: AI safety community building and research in South Africa

Leaders: Leo Hyams and Benjamin Sturgeon

Funding Needed: Low

Confidence Level: Low

This is a mix of AI research and building up the local AI safety community. One person whose opinion I value gave the plan and those involved in it a strong endorsement, so including it based on that.

To donate, reach out to [email protected].

Focus: Incubation, fellowship and training in India for technical AI safety

Leader: Sebastian Schmidt

Funding Needed: Medium

Confidence Level: Low

I buy the core idea that India is a place to get good leverage on a lot of underserved talent, that is not going to otherwise get exposure to AI safety ideas and potentially not get other good opportunities either, all on the cheap. So this makes a lot of sense.

To donate, contact [email protected].

Focus: Fellowships and affiliate programs for new alignment researchers

Leader: Nora Ammann, Lucas Teixeira and Dusan D. Nesic

Funding Needed: High

Confidence Level: Low

The same logic applies here as applies to the other talent funnels. It seems like a solid talent funnel when I look at who they sent through it, and one other recommender thought their approach was strong, but do we need more of this? If you think we straightforwardly need more help with people starting out in alignment work, then this is a solid place to look.

To donate, reach out to [email protected].

Focus: Journalism fellowships for oversight of AI companies.

Leader: Cillian Crosson (same as Talos Network for now but she plans to focus here)

Funding Needed: Medium

Confidence Level: Low

They offer fellowships to would-be journalists so they can out and provide ‘democratic oversight of AI.’ They have sponsored at least one person who went on to do good work in the area.

I am not sure this is a place we need to do more investment, or if people trying to do this even need fellowships. Hard to say. There’s certainly a lot more tech reporting and more every day, if I’m ever short of material I have no trouble finding more. It is still a small amount of money per person that can meaningfully help people get on their feet and do something useful. We do in general need better journalism.

Focus: Incubation of AI safety organizations

Leader: Alexandra Bos

Funding Needed: Low

Confidence Level: Low

Why funnel individual talent when you can incubate entire organizations? I am not convinced that on the margin we currently need more of either, but I’m more receptive to the idea of an incubator. Certainly incubators can be high leverage points for getting valuable new orgs and companies off the ground, especially if your model is that once the org becomes fundable it can unlock additional funding. And the price is right, so this could be worth a shot even if we’re somewhat saturated on orgs already, to try and get better ones. If you think an incubator is worth funding, then the question is whether this is the right team. The application was solid all around, but beyond that I don’t have a differentiator on why this is the team.

Focus: Various field building activities in AI safety

Leader: Victoria Brook

Funding Needed: Medium

Confidence Level: Low

Kind of an 80,000 hours variant. They also help find funding and compute and make connections, and offer 30 minute phone calls. Their job board seems seems like a useful thing and passes at least some sanity checks on what not to list, I’ve referenced it before in the newsletter. Highly plausible choice if that fits your investment thesis.

To donate, reach out to [email protected].

Focus: New AI safety org in Paris, discourse, R&D collaborations, talent pipeline

Leaders: Charbel-Raphael Segerie, Florent Berthet

Funding Needed: Medium

Confidence Level: Low

They’re doing all three of discourse, direct work and talent funnels. I put them in the talent section based on my read of where they have biggest emphasis and best case for existing impact. I see enough social proof of them doing the things that I’m happy to list them, in case people are excited to back a new org of this type.

To donate, go here.

Focus: Recruitment for existential risk causes

Leader: Steve Luby and Paul Edwards

Funding Needed: Medium

Confidence Level: Low

Stanford students certainly are one place to find people worth educating about existential risk. It’s also an expensive place to be doing it, and a place that shouldn’t need extra funding. And that hates fun. And it’s not great that AI is listed third on their existential risk definition. So I’m not high on them, but it sure beats giving unrestricted funds to your Alma Mater.

Interested donors should contact Steve Luby directly at [email protected].

And that’s a wrap!

If an organization was not included here, again, that does not mean they aren’t good, or even that I wouldn’t endorse them if asked. It could be because they didn’t apply to SFF, or because I didn’t give them the time and attention they need, or in several cases because I wrote up a section for them but they asked to be excluded – if by accident I included you and you didn’t want to be included and I failed to remove you, or you don’t like the quote here, I sincerely apologize and will edit you out right away, no questions asked.

If an organization is included here, that is a good thing, but again, it does not mean you should donate without checking if it makes sense based on what you think is true, how you think the world works, what you value and what your priorities are. There are no universal right answers.

The Big Nonprofits Post Read More »

What fossilized dino feces can tell us about their rise to dominance

coprolites, dinosaurs, evolutionary biology, food webs, fossilized feces, Jurassic, paleontology, Science, Triassic / Blake Jones / November 27, 2024

Paleontologists have long puzzled over how the dinosaurs—originally relatively small and of minor importance to the broader ecosystem—evolved to become the dominant species some 30 million years later. Fossilized feces and vomit from dinosaurs might hold important clues to how and why this evolutionary milestone came about, according to a new paper published in the journal Nature.

Co-author Martin Qvarnström, an evolutionary biologist with Uppsala University in Sweden, and his collaborators studied trace fossils known as bromalites, a designation that includes coprolites as well as vomit or other fossilized matter from an organism’s digestive tract. As previously reported, coprolites aren’t quite the same as paleofeces, which retain a lot of organic components that can be reconstituted and analyzed for chemical properties. Coprolites are fossils, so most organic components have been replaced by mineral deposits like silicate and calcium carbonates.

For archaeologists keen on learning more about the health and diet of past populations—as well as how certain parasites evolved in the evolutionary history of the microbiome—coprolites and paleofeces can be a veritable goldmine of information. For instance, in 2021 we reported on an analysis of preserved paleo-poop revealing that ancient Iron Age miners in what is now Austria were fond of beer and blue cheese.

If a coprolite contains bone fragments, chances are the animal who excreted it was a carnivore, and tooth marks on those fragments can tell us something about how the animal may have eaten its prey. The size and shape of coprolites can also yield useful insights. If a coprolite is spiral-shaped, for instance, it might have been excreted by an ancient shark, since some modern fish (like sharks) have spiral-shaped intestines.

A tale of two models

Excavations in the Late Triassic locality at Lisowice, Poland. The site yielded a large number of coprolites of predators and herbivores. Credit: Krystian Balanda

Qvarnström et al. were keen to test two competing hypotheses about the dinosaurs’ rise to dominance from the Late Triassic Period (237 million to 201 million years ago) to the onset of the Jurassic Period between 201 million to 145 million years ago. “No single hypothesis seems capable of explaining the rise of dinosaurs fully and critical questions about how dinosaurs established their dynasty on land remain largely unanswered,” the authors wrote about their research objectives.

One hypothesis cites evolutionary competition—the traditional “competitive replacement” model—as a driving factor, in which dinosaurs were better equipped to survive thanks to superior physiologies, anatomical adaptations, and feeding habits. Alternatively the “opportunistic replacement” model suggests that the dinosaurs were better able to adapt to a rapidly changing environment brought about by random processes—volcanic eruptions, climate change, or other catastrophic events that led to the decline and/or extinction of other species.

What fossilized dino feces can tell us about their rise to dominance Read More »

Things aren’t looking good for infamous CEO of “health care terrorists”

corruption, health, health care, health care industry, hospitals, Malta, ralph de la torre, Science, steward / Blake Jones / November 26, 2024

Earlier this year, a Maltese magistrate concluded a four-year investigation into the matter and recommended that Ernst and de la Torre be charged with money laundering, criminal association, and corruption of public officials, including the nation’s former prime minister, Joseph Muscat, the Globe reports.

Meanwhile, new allegations of domestic dealings continue to come to light. In a separate investigative story Monday, the Globe reported that Steward executives used Steward-owned malpractice insurer TRACO “like a piggy bank.” The Panama-based TRACO was supposed to work like an independent insurer for the hospital chain; Steward would pay TRACO malpractice insurance premiums on behalf of its doctors and the pooled money would be used to litigate and pay out claims. But, instead of paying premiums, Steward gave TRACO IOUs. By the end of 2023, TRACO’s accounting records showed $99 million in outstanding loans, most owed by Steward, and $176 million in “accounts receivable,” also mostly owed by Steward.

With Steward now in bankruptcy, insurance coverage for health care providers is now in question, as are payouts to patients who were harmed by Steward’s care. The Globe noted the case of Yasmany Sosa, whose 35-year-old wife, Yanisey Rodriguez, died a preventable death after giving birth at Steward North Shore Medical Center in Florida in September 2022. Steward agreed to a $4 million settlement with Sosa in March, but the money hasn’t appeared, leaving Sosa in limbo and struggling.

“They killed my wife, that’s for starters. Second of all, they destroyed my family,” Sosa told the Globe through a translator. “This has all become a bunch of loopholes, legal strategies. This really is very difficult for me… I’ve already lost everything.”

Things aren’t looking good for infamous CEO of “health care terrorists” Read More »

Author name: Blake Jones

Along came a spider

The common garden experiment

Underwater forests

Sharing seeds

Downpour of disease

Plan unlikely to survive Trump administration

Trump DOGE appointee: “Delete the CFPB”

304 million Tesla shares

A tale of two models