Federal civilian agencies have until midnight Saturday morning to sever all network connections to Ivanti VPN software, which is currently under mass exploitation by multiple threat groups. The US Cybersecurity and Infrastructure Security Agency mandated the move on Wednesday after disclosing three critical vulnerabilities in recent weeks.
Three weeks ago, Ivanti disclosed two critical vulnerabilities that it said threat actors were already actively exploiting. The attacks, the company said, targeted “a limited number of customers” using the company’s Connect Secure and Policy Secure VPN products. Security firm Volexity said on the same day that the vulnerabilities had been under exploitation since early December. Ivanti didn’t have a patch available and instead advised customers to follow several steps to protect themselves against attacks. Among the steps was running an integrity checker the company released to detect any compromises.
Almost two weeks later, researchers said the zero-days were under mass exploitation in attacks that were backdooring customer networks around the globe. A day later, Ivanti failed to make good on an earlier pledge to begin rolling out a proper patch by January 24. The company didn’t start the process until Wednesday, two weeks after the deadline it set for itself.
And then, there were three
Ivanti disclosed two new critical vulnerabilities in Connect Secure on Wednesday, tracked as CVE-2024-21888 and CVE-2024-21893. The company said that CVE-2024-21893—a class of vulnerability known as a server-side request forgery—“appears to be targeted,” bringing the number of actively exploited vulnerabilities to three. German government officials said they had already seen successful exploits of the newest one. The officials also warned that exploits of the new vulnerabilities neutralized the mitigations Ivanti advised customers to implement.
Hours later, the Cybersecurity and Infrastructure Security Agency—typically abbreviated as CISA—ordered all federal agencies under its authority to “disconnect all instances of Ivanti Connect Secure and Ivanti Policy Secure solution products from agency networks” no later than 11: 59 pm on Friday. Agency officials set the same deadline for the agencies to complete the Ivanti-recommended steps, which are designed to detect if their Ivanti VPNs have already been compromised in the ongoing attacks.
The steps include:
Identifying any additional systems connected or recently connected to the affected Ivanti device
Monitoring the authentication or identity management services that could be exposed
Isolating the systems from any enterprise resources to the greatest degree possible
Continuing to audit privilege-level access accounts.
The directive went on to say that before agencies can bring their Ivanti products back online, they must follow a long series of steps that include factory resetting their system, rebuilding them following Ivanti’s previously issued instructions, and installing the Ivanti patches.
“Agencies running the affected products must assume domain accounts associated with the affected products have been compromised,” Wednesday’s directive said. Officials went on to mandate that by March 1, agencies must have reset passwords “twice” for on-premise accounts, revoke Kerberos-enabled authentication tickets, and then revoke tokens for cloud accounts in hybrid deployments.
Steven Adair, the president of Volexity, the security firm that discovered the initial two vulnerabilities, said its most recent scans indicate that at least 2,200 customers of the affected products have been compromised to date. He applauded CISA’s Wednesday directive.
“This is effectively the best way to alleviate any concern that a device might still be compromised,” Adair said in an email. “We saw that attackers were actively looking for ways to circumvent detection from the integrity checker tools. With the previous and new vulnerabilities, this course of action around a completely fresh and patched system might be the best way to go for organizations to not have to wonder if their device is actively compromised.”
The directive is binding only on agencies under CISA’s authority. Any user of the vulnerable products, however, should follow the same steps immediately if they haven’t already.
Enlarge/ Booking photo of Alan Filion, charged with multiple felonies connected to a “swatting” incident at the Masjid Al Hayy Mosque in Sanford, Florida.
Police suspect that a 17-year-old from California, Alan Filion, may be responsible for “hundreds of swatting incidents and bomb threats” targeting the Pentagon, schools, mosques, FBI offices, and military bases nationwide, CNN reported.
Swatting occurs when fraudulent calls to police trigger emergency response teams to react forcefully to non-existent threats.
Recently extradited to Florida, Filion was charged with multiple felonies after the Seminole County Sheriff’s Office (SCSO) traced a call where Filion allegedly claimed to be a mass shooter entering the Masjid Al Hayy Mosque in Sanford, Florida. The caller played “audio of gunfire in the background,” SCSO said, while referencing Satanism and claiming he had a handgun and explosive devices.
Approximately 30 officers responded to the call in May 2023, then determined it was a swatting incident after finding no shooter and confirming that mosque staff was safe. In a statement, SCSO Sheriff Dennis Lemma said that “swatting is a perilous and senseless crime, which puts innocent lives in dangerous situations and drains valuable resources” by prompting a “substantial law enforcement response.”
Seminole County authorities coordinated with the FBI and Department of Justice to track the alleged “serial swatter” down, ultimately arresting Filion on January 18. According to SCSO, police were able to track down Filion after he allegedly “created several accounts on websites offering swatting services” that were linked to various IP addresses connected to his home address. The FBI then served a search warrant on the residence and found “incriminating evidence.”
Filion has been charged as an adult for a variety of offenses, including making a false report while facilitating or furthering an act of terrorism. He is currently being detained in Florida, CNN reported.
Earlier this year, Sen. Rick Scott (R-Fla.) introduced legislation to “crack down” on swattings after he became a target at his home in December. If passed, the Preserving Safe Communities by Ending Swatting Act would impose strict penalties, including a maximum sentence of 20 years in prison for any swatting that lead to serious injuries. If death results, bad actors risk a lifetime sentence. That bill is currently under review by the House Judiciary Committee.
“We must send a message to the cowards behind these calls—this isn’t a joke, it’s a crime,” Scott said.
Last year, Sen. Chuck Schumer (D-NY) warned that an “unprecedented wave” of swatting attacks in just two weeks had targeted 11 states, including more than 200 schools across New York. In response, Schumer called for over $10 million in FBI funding to “specifically tackle the growing problem of swatting.”
Schumer said it was imperative that the FBI begin tracking the incidents more closely, not just to protect victims from potentially deadly swattings, but also to curb costs to law enforcement and prevent unnecessary delays of emergency services tied up by hoax threats.
As a result of Schumer’s push, the FBI announced it would finally begin tracking swatting incidents nationwide. Hundreds of law enforcement agencies and police departments now rely on an FBI database to share information on swatting incidents.
Coordination appears to be key to solving these cases. Lemma noted that SCSO has an “unwavering dedication” to holding swatters accountable, “regardless of where they are located.” His office confirmed that investigators suspect that Filion may have also been behind “other swatting incidents” across the US. SCSO said that it will continue coordinating with local authorities investigating those incidents.
“Make no mistake, we will continue to work tirelessly in collaboration with our policing partners and the judiciary to apprehend swatting perpetrators,” Lemma said. “Gratitude is extended to all agencies involved at the local, state, and federal levels, and this particular investigation and case stands as a stern warning: swatting will face zero tolerance, and measures are in place to identify and prosecute those responsible for such crimes.”
The Federal Communications Commission plans to vote on making the use of AI-generated voices in robocalls illegal. The FCC said that AI-generated voices in robocalls have “escalated during the last few years” and have “the potential to confuse consumers with misinformation by imitating the voices of celebrities, political candidates, and close family members.”
FCC Chairwoman Jessica Rosenworcel’s proposed Declaratory Ruling would rule that “calls made with AI-generated voices are ‘artificial’ voices under the Telephone Consumer Protection Act (TCPA), which would make voice cloning technology used in common robocalls scams targeting consumers illegal,” the commission announced yesterday. Commissioners reportedly will vote on the proposal in the coming weeks.
A recent anti-voting robocall used an artificially generated version of President Joe Biden’s voice. The calls told Democrats not to vote in the New Hampshire Presidential Primary election.
An analysis by the company Pindrop concluded that the artificial Biden voice was created using a text-to-speech engine offered by ElevenLabs. That conclusion was apparently confirmed by ElevenLabs, which reportedly suspended the account of the user who created the deepfake.
FCC ruling could help states crack down
The TCPA, a 1991 US law, bans the use of artificial or prerecorded voices in most non-emergency calls “without the prior express consent of the called party.” The FCC is responsible for writing rules to implement the law, which is punishable with fines.
As the FCC noted yesterday, the TCPA “restricts the making of telemarketing calls and the use of automatic telephone dialing systems and artificial or prerecorded voice messages.” Telemarketers are required “to obtain prior express written consent from consumers before robocalling them. If successfully enacted, this Declaratory Ruling would ensure AI-generated voice calls are also held to those same standards.”
The FCC has been thinking about revising its rules to account for artificial intelligence for at least a few months. In November 2023, it launched an inquiry into AI’s impact on robocalls and robotexts.
Rosenworcel said her proposed ruling will “recognize this emerging technology as illegal under existing law, giving our partners at State Attorneys General offices across the country new tools they can use to crack down on these scams and protect consumers.
“AI-generated voice cloning and images are already sowing confusion by tricking consumers into thinking scams and frauds are legitimate,” Rosenworcel said. “No matter what celebrity or politician you favor, or what your relationship is with your kin when they call for help, it is possible we could all be a target of these faked calls.”
Google has another fix for the second major storage bug Pixel phones have seen in the last four months. Last week, reports surfaced that some Pixel owners were being locked out of their phone’s local storage, creating a nearly useless phone with all sorts of issues. Many blamed the January 2024 Google Play system update for the issue, and yesterday, Google confirmed that hypothesis. Google posted an official solution to the issue on the Pixel Community Forums, but there’s no user-friendly solution here. Google’s automatic update system broke people’s devices, but the fix is completely manual, requiring users to download the developer tools, install drivers, change settings, plug in their phones, and delete certain files via a command-line interface.
The good news is that, if you’ve left your phone sitting around in a nearly useless state for the last week or two, following the directions means you won’t actually lose any data. Having a week or two of downtime is not acceptable to a lot of people, though, and several users replied to the thread saying they had already wiped their device to get their phone working again and had to deal with the resulting data loss (despite many attempts and promises, Android does not have a comprehensive backup system that works).
The bad news is that I don’t think many normal users will be able to follow Google’s directions. First, you’ll need to perform the secret action to enable Android’s Developer Options (you tap on the build number seven times). Then, you have to download Google’s “SDK Platform-Tools” zip file, which is meant for app developers. After that, plug in your phone, switch to the correct “File transfer” connection mode, open a terminal, navigate to the platform-tools folder, and run both “./adb uninstall com.google.android.media.swcodec” and “./adb uninstall com.google.android.media.” Then reboot the phone and hope that works.
I skipped a few steps (please read Google’s instructions if you’re trying this), but that’s the basic gist of it. The tool Google is having people use is “ADB,” or the “Android Debug Bridge.” This is meant to give developers command-line access to their phones, which allows them to quickly push new app builds to the device, get a readout of system logs, and turn on special developer flags for various testing.
Google’s instructions will only work if everything goes smoothly, and as someone with hundreds of hours in ADB from testing various Android versions, I will guess that it will probably not go smoothly. On Windows, the ADB drivers often don’t install automatically. Instead, you’ll get “unknown device” or some other incorrect device detection, and you won’t be able to run any commands. You usually have to use the “let me pick from drivers on my computer” option, browse through your file system, and manually “select” (more like “guess”) the driver you need while clicking through various warnings. You can already see at least one user with driver issues in the thread, with Windows telling them, “Your device has malfunctioned,” when really it just needs a driver.
Enlarge / Selena Gomez and Martin Short on the set of Only Murders in the Building on February 14, 2022, in New York City.
Hulu and Disney+ subscribers have until March 14 to stop sharing their login information with people outside of their household. Disney-owned streaming services are the next to adopt the password-crackdown strategy that has helped Netflix add millions of subscribers.
An email sent from “The Hulu Team” to subscribers yesterday and viewed by Ars Technica tells customers that Hulu is “adding limitations on sharing your account outside of your household.”
Hulu’s subscriber agreement, updated on January 25, now states that users “may not share your subscription outside of your household,” with household being defined as the “collection of devices associated with your primary personal residence that are used by the individuals who reside therein.”
The updated terms also note that Hulu might scrutinize user accounts to ensure that the accounts aren’t being used on devices located outside of the subscriber’s residence:
We may, in our sole discretion, analyze the use of your account to determine compliance with this Agreement. If we determine, in our sole discretion, that you have violated this Agreement, we may limit or terminate access to the Service and/or take any other steps as permitted by this Agreement (including those set forth in Section 6 of this Agreement).
Section 6 of Hulu’s subscriber agreement says Hulu can “restrict, suspend, or terminate” access without notice.
Hulu didn’t respond to a request for comment on how exactly it will “analyze the use” of accounts. But Netflix, which started its password crackdown in March 2022 and brought it to the US in May 2023, says it uses “information such as IP addresses, device IDs, and account activity to determine whether a device signed in to your account is part of your Netflix Household” and doesn’t collect GPS data from devices.
According to the email sent to Hulu subscribers, the policy will apply immediately to people subscribing to Hulu from now on.
The updated language in Hulu’s subscriber agreement matches what’s written in the Disney+/ESPN+ subscriber agreement, which was also updated on January 25. Disney+’s password crackdown first started in November in Canada.
A Disney spokesperson confirmed to Ars Technica that Disney+ subscribers have until March 14 to comply. The rep also said that notifications were sent to Disney+’s US subscribers yesterday; although, it’s possible that some subscribers didn’t receive an email alert, as is the case with a subscriber in my household.
The representative didn’t respond to a question asking how Disney+ will “analyze” user accounts to identify account sharing.
Push for profits
Disney CEO Bob Iger first hinted at a Disney streaming-password crackdown in August during an earnings call. He highlighted a “significant” amount of password sharing among Disney-owned streaming services and said Disney had “the technical capability to monitor much of this.” The executive hopes a password crackdown will help drive subscribers and push profits to Netflix-like status. Disney is aiming to make its overall streaming services business profitable by the end of 2024.
In November, it was reported that Disney+ had lost $11 billion since launching in November 2019. The streaming service has sought to grow revenue by increasing prices and encouraging users to join its subscription tier with commercials, which is said to bring streaming services higher average revenue per user (ARPU) than non-ad plans.
Hulu, which Disney will soon own completely, has been profitable in the past, and in Disney’s most recent financial quarter, it had a higher monthly ARPU than Disney+. Yet, Hulu has far fewer subscribers than Disney+ (48.5 million versus 150.2 million). Cracking down on Hulu password sharing is an obvious way for Disney to try to squeeze more money from the more financially successful streaming service.
Such moves run the risk of driving away users. However, Hulu, like Netflix, may be able to win over longtime users who have gotten accustomed to having easy access to Hulu, even if they weren’t paying for it. Disney+, meanwhile, is a newer service, so a change in policy may not feel as jarring to some.
Netflix, which allowed account sharing for years, has seen success with its password crackdown, saying in November that the efforts helped it add 8.8 million subscribers. Unlike the Disney-owned streaming services, though, Netflix allows people to add extra members to their non-ad subscription (in the US, Netflix charges $7.99 per person per month).
As Disney embarks on an uphill climb to make streaming successful this year, you can expect it to continue following the leader while also trying to compete with it. Around the same time as the password-sharing ban takes full effect, Disney should also unveil a combined Hulu-Disney+ app, a rare attempt at improving a streaming service that doesn’t center on pulling additional monthly dollars from customers.
Enlarge/ An artist’s concept of the Starlab space station.
Starlab LLC
The Starlab commercial space station will launch on SpaceX’s Starship rocket, officials said this week.
Starlab is a joint venture between the US-based Voyager Space and the European-based multinational aerospace corporation Airbus. The venture is building a large station with a habitable volume equivalent to half the pressurized volume of the International Space Station and will launch the new station no earlier than 2028.
“SpaceX’s history of success and reliability led our team to select Starship to orbit Starlab,” Dylan Taylor, chairman and CEO of Voyager Space, said in a statement. “SpaceX is the unmatched leader for high-cadence launches and we are proud Starlab will be launched to orbit in a single flight by Starship.”
Fitting in a big fairing
Starlab will have a diameter of about 26 feet (8 meters). It is perhaps not a coincidence that Starship’s payload bay can accommodate vehicles up to 26 feet across in its capacious fairing. However, in an interview, Marshall Smith, the chief technology officer of Voyager Space, said the company looked at a couple of launch options.
“We looked at multiple launches to get Starlab into orbit, and eventually gravitated toward single launch options,” he said. “It saves a lot of the cost of development. It saves a lot of the cost of integration. We can get it all built and checked out on the ground, and tested and launch it with payloads and other systems. One of the many lessons we learned from the International Space Station is that building and integrating in space is very expensive.”
With a single launch on a Starship, the Starlab module should be ready for human habitation almost immediately, Smith said.
Enlarge/ It’s hard to believe the interior of Starlab will ever be this clean in space.
Starlab LLC
Starlab is one of several privately developed space stations vying to become a commercial replacement for the International Space Station, which NASA is likely to retire in 2030. Among the other contenders are Axiom Space, Blue Origin, and Vast Space. SpaceX may also configure a human-rated version of Starship as a temporary space station.
NASA has provided seed funding to some of these companies, including Voyager Space, to begin designing and developing their stations. NASA is expected to hold a second round of competition next year, when it will select one or more companies to proceed with building and testing their stations.
Finding customers
Each company is developing a space station that will serve both government customers—NASA wants to continue flying at least a handful of astronauts in low-Earth orbit for research purposes—as well as private customers. The challenge for Starlab and other commercial stations is developing a customer base beyond NASA to support the expense of flying and operating stations.
The challenge is a huge one: NASA spent more than $100 billion constructing the International Space Station and has a $3 billion annual budget for operations and transportation of people and supplies to the station. The agency is likely to fund commercial space stations at a level of about $1 billion a year, so these companies must build their facilities relatively quickly at low cost and then find a diverse base of customers to offset expenses.
Starlab may have an advantage in this regard with its co-ownership by Airbus. One of the big questions surrounding the end of the International Space Station is what happens to the European astronauts who fly there now. The European Space Agency will likely be reticent about funding missions to private space stations owned and operated by US companies. The involvement by Airbus, therefore, makes Starlab attractive to European nations as a destination.
Enlarge/ Clown anemonefish (Amphiprion ocellaris) seem to recognize different species of clownfish by counting white stripes.
Kina Hayashi
Many people tend to think of clownfish, with their distinctive white bars against an orange, red, or black background, as a friendly sort of fish, perhaps influenced to some extent by the popular Pixar film Finding Nemo. But clownfish can be quite territorial when it comes to defending their host anemone from intrusion by others, particularly those from their own species. A new paper published in the Journal of Experimental Biology describes how clownfish determine if a fish approaching their home is friend or foe by “counting” the number of white bars or stripes on their bodies.
As previously reported, mathematical ability is often considered uniquely human, but in fact, scientists have found that many animal species—including lions, chimpanzees, birds, bees, ants, and fish—seem to possess at least a rudimentary counting ability or number sense. Crows can understand the concept of zero. So can bees, which can also add and subtract, as can both stingrays and cichlids—at least for a small number of objects (in the range of one to five). Some ants count their steps.
This so-called “numerosity” simply refers to the number of things in a set, according to cognitive psychologist Brian Butterworth, an emeritus professor at University College London and author of Can Fish Count? What Animals Reveal About Our Uniquely Mathematical Minds. It has nothing to do with reasoning or logical mathematical intelligence. This is information that will be in the environment, and counting animals must have some mechanism for extracting this numerical information from the environment. But it nonetheless makes for a fascinating field of study.
In 2022, Kina Hayashi of the Okinawa Institute of Science and Technology (OIST) and several colleagues found that clownfish display more aggressive behavior (e.g., chasing or biting) toward fish (or fish toys) with vertical bar patterns compared with fish with horizontal stripe patterns and that this aggressive behavior lasted longer when directed at fish with vertical bars versus horizontal bars. This behavior appears to influence the position of fish species between host anemones and coral reefs: No fish with vertical bars sought shelter in host anemones, while several species with vertical bars were found in the surrounding coral reefs. But it wasn’t clear how the fish recognized the color patterns or what basic rules controlled this signaling. The study results suggested that it wasn’t based on the mere presence of white bars or how much white color was present on a given fish’s body.
Enlarge/ The plastic models used to measure the clown anemonefish’s aggressive behavior.
This new study builds on that earlier work. This time around, Kayashi and co-authors raised a school of young common clownfish (A. ocellaris) from eggs to ensure that the fish had never set eyes on other species of anemonefish. At six months old, the fish were introduced to several other clownfish species, including Clarke’s anemonefish (A. clarkii), orange skunk clownfish (A. sandaracinos), and saddleback clownfish (A. polymnus).
The researchers placed different species of clownfish, with different numbers of white bars, in small cases inside a tank with a clownfish colony and filmed their reaction. Because they were in a controlled tank environment, there was no chasing or biting. Rather, aggressive behavior was defined as staring aggressively at the other fish and circling the case in which the other fish were held.
They followed up with a second set of experiments in which they presented a colony of clownfish with different plastic models painted with accurate clownfish coloration, with differing numbers of white stripes. The researchers also filmed and measured the degree of aggressive behavior directed at the different plastic models.
Clownfish showing aggression toward another fish with similar stripes. Credit: Kina Hayashi
The results: “The frequency and duration of aggressive behaviors in clown anemonefish was highest toward fish with three bars like themselves,” said Hayashi, “while they were lower with fish with one or two bars, and lowest toward those without vertical bars, which suggests that they are able to count the number of bars in order to recognize the species of the intruder.”
Hayashi et al. cautioned that one limitation of their study is that all the fish used in the experiments were hatched and raised in an environment where they had only encountered other fish of their own species. So, they could not conclusively determine whether the observed behavior was innate or learned. Other species of clownfish also use the same anemone species as hosts, so aggressive behavior toward those species might be more frequent in the wild than observed in the laboratory tank environment.
The unexpected success of Palworld continues to be one of the biggest gaming stories of 2024 so far, as developer Pocketpair says the game’s sales and Xbox downloads have exceeded 19 million, with 12 million in sales on Steam and 7 million players on Xbox. Microsoft has also announced that the game has been the biggest third-party launch in the Game Pass service’s history, as well as the most-played third-party title on the Xbox Cloud Gaming service.
These numbers continue a remarkable run for the indie-developed Pokémon-survival-crafting-game pastiche, which sold 5 million copies in its first weekend as a Steam Early Access title and had sold 8 million Steam copies as of a week ago. There are signs that the game’s sales are slowing down—it’s currently Steam’s #2 top-selling game after over a week in the #1 spot. But its active player count on Steam remains several hundred thousand players higher than Counter-Strike 2, the next most-played game on the platform.
Sometimes described (both admiringly and disparagingly) as “Pokémon with guns,” Palworld‘s unexpected success has driven some Internet outrage cycles about the possibility that it may have used AI-generated monster designs and allegations that its designers copied or modified some of the 3D character models from the actual Pokémon series to create some of the game’s more familiar-looking monsters.
The latter allegations circulated widely enough that The Pokémon Company issued a statement last week, saying it would “investigate” an unnamed game that matches Palworld‘s description; as of this writing, no actual legal action has been taken against Palworld or Pocketpair. Third-party modders who have tried to put actual Pokémon creatures into Palworld have apparently gotten some cease-and-desist letters, though.
Regardless, the game wears its influences on its sleeve. Aside from the Pals that look like Pokémon, the game’s progression and crafting mechanics owe a lot to games like ARK: Survival Evolved, and the actual monster-catching mechanics have a more-than-passing resemblance to Pokémon Legends: Arceus.
If you count the Xbox Game Pass numbers as “sales,” Palworld‘s combined numbers are on track to overtake those of the two main-series Pokémon titles on the Nintendo Switch, Sword/Shield and Scarlet/Violet. Nintendo says these games have sold 26.02 and 23.23 million copies, respectively, making them the sixth and seventh bestselling titles in the entire Switch library.
Nintendo doesn’t break out sales figures for each title individually, counting each sale of Sword or Shield toward the same total—this makes sense because they’re the same basic game with slightly different Pokémon, though it does mean there’s some double-dipping going on for fans who buy both versions of a given game for themselves. You have to look at proxies like Amazon reviews to get a sense of which individual version has sold better—Violet currently has more reviews than Scarlet, while Sword has more reviews than Shield.
Enlarge/ Tesla CEO Elon Musk speaks at Tesla’s “Cyber Rodeo” on April 7, 2022, in Austin, Texas.
Getty Images | AFP/Suzanne Cordeiro
Tesla CEO Elon Musk has had enough of Delaware after a state court ruling voided his $55.8 billion pay package. Musk said last night that Tesla will hold a shareholder vote on transferring the electric carmaker’s state of incorporation to Texas.
Musk had posted a poll on X (formerly Twitter) asking whether Tesla should “change its state of incorporation to Texas, home of its physical headquarters.” After over 87 percent of people voted yes, Musk wrote, “The public vote is unequivocally in favor of Texas! Tesla will move immediately to hold a shareholder vote to transfer state of incorporation to Texas.”
Tesla was incorporated in 2003 before Musk joined the company. Its founders chose Delaware, a common destination because of the state’s low corporate taxes and business-friendly legal framework. The Delaware government says that over 68 percent of Fortune 500 companies are registered in the state, and 79 percent of US-based initial public offerings in 2022 were registered in Delaware.
One reason for choosing Delaware is the state’s Court of Chancery, where cases are decided not by juries but by judges who specialize in corporate law. On Tuesday, Court of Chancery Judge Kathaleen McCormick ruled that Musk’s $55.8 billion pay package was unfair to shareholders and must be rescinded.
McCormick’s ruling in favor of the plaintiff in a shareholder lawsuit said that most of Tesla’s board members “were beholden to Musk or had compromising conflicts.” McCormick also concluded that the Tesla board gave shareholders inaccurate and misleading information in order to secure approval of Musk’s “unfathomable” pay plan.
Musk a fan of Texas and Nevada
Musk yesterday shared a post claiming that McCormick’s ruling “is another clear example of the Biden administration and its allies weaponizing the American legal system against their political opponents.”
McCormick previously oversaw the Twitter lawsuit that forced Musk to complete a $44 billion purchase despite his attempt to break a merger agreement. After Musk became Twitter’s owner, he merged the company into X Corp., which is registered in Nevada.
“Never incorporate your company in the state of Delaware,” Musk wrote in a post after the Delaware court ruling. “I recommend incorporating in Nevada or Texas if you prefer shareholders to decide matters,” he also wrote.
Last year, Texas enacted a law to create business courts that will hear corporate cases. The courts are slated to begin operating on September 1, 2024. Musk is clearly hoping the new Texas courts will be more deferential to Tesla on executive pay if the company is sued again after his next pay plan is agreed on.
Tesla shareholders who will be asked to vote on a corporate move to Texas “need to take a hard look at how transitioning out of Delaware might impact their rights and the company’s governance,” Reuters quoted business adviser Keith Donovan as saying.
Reuters quoted AJ Bell investment analyst Dan Coatsworth as saying that “Elon Musk’s plan to change Tesla’s state of incorporation from Delaware to Texas is typical behavior for the entrepreneur who always looks for an alternative if he can’t get what he wants.”
Two studies came out on the question of whether existing LLMs can help people figure out how to make bioweapons. RAND published a negative finding, showing no improvement. OpenAI found a small improvement, bigger for experts than students, from GPT-4. That’s still harmless now, the question is what will happen in the future as capabilities advance.
Another news item was that Bard with Gemini Pro impressed even without Gemini Ultra, taking the second spot on the Arena leaderboard behind only GPT-4-Turbo. For now, though, GPT-4 remains in the lead.
A third cool item was this story from a Russian claiming to have used AI extensively in his quest to find his one true love. I plan to cover that on its own and have Manifold on the job of figuring out how much of the story actually happened.
Introduction.
Table of Contents.
Language Models Offer Mundane Utility. Bard is good now even with only Pro?
Language Models Don’t Offer Mundane Utility. Thinking well remains hard.
GPT-4 Real This Time. Bring GPTs into normal chats, cheaper GPT-3.5
Be Prepared. How much can GPT-4 enable production of bioweapons?
Fun With Image Generation. How to spot an AI image, new MidJourney model.
Deepfaketown and Botpocalypse Soon. Taylor Swift fakes, George Carlin fake.
They Took Our Jobs. If they did, how would we know?
Get Involved. What we have here is a failure to communicate.
In Other AI News. Who is and is not raising capital or building a team.
Quiet Speculations. Is economic growth caused by inputs or by outputs?
The Quest for Sane Regulation. Emergency emergency emergency. Meh.
According to the system card, roughly, Gemini Ultimate is to Gemini Pro as GPT-4 is to GPT-3.5. If that is true, this is indeed evaluated on Gemini Pro and Bard gets a similar Elo boost as ChatGPT does when it leaps models, then the version of Bard with Gemini Ultimate could clock in around 1330, the clear best model. Your move, OpenAI.
Some of the comments were suspicious that they were somehow getting Gemini Ultimate already, or that Bard was doing this partly via web access, or that it was weird that it could score that high given some rather silly refusals and failures. There are clearly places where Bard falls short. There is also a lot of memory of when Bard was in many ways much worse than it is now, and lack of knowledge of the places Bard is better.
Ethan Mollick uses prompt engineering and chain of thought to get GPT-4 to offer ‘creative ideas’ for potential under $50 products for college students in a new paper. The claim is that without special prompts the ideas are not diverse, but with prompting and CoT this can largely be fixed.
I put creative ideas in air quotes because the thing that Ethan consistently describes as creativity, that he says GPT-4 is better at than most humans, does not match my central understanding of creativity.
Here is the key technique and result:
Exhaustion
We picked our most successful strategy (Chain of Thought) and compared it against the base strategy when generating up to 1200 ideas in one session. We used the following prompts:
Base Prompt
Generate new product ideas with the following requirements: The product will target college students in the United States. It should be a physical good, not a service or software. I’d like a product that could be sold at a retail price of less than about USD 50.
The ideas are just ideas. The product need not yet exist, nor may it necessarily be clearly feasible. Number all ideas and give them a name. The name and idea are separated by a colon. Please generate 100 ideas as 100 separate paragraphs. The idea should be expressed as a paragraph of 40-80 words.
Chain of Thought
Generate new product ideas with the following requirements: The product will target college students in the United States. It should be a physical good, not a service or software. I’d like a product that could be sold at a retail price of less than about USD 50.
The ideas are just ideas. The product need not yet exist, nor may it necessarily be clearly feasible.
Follow these steps. Do each step, even if you think you do not need to.
First generate a list of 100 ideas (short title only)
Second, go through the list and determine whether the ideas are different and bold, modify the ideas as needed to make them bolder and more different. No two ideas should be the same. This is important!
Next, give the ideas a name and combine it with a product description. The name and idea are separated by a colon and followed by a description. The idea should be expressed as a paragraph of 40-80 words. Do this step by step!
Note that on some runs, the model did not properly follow the second step and deemed the ideas bold enough without modification. These runs have been removed from the final aggregation (around ~15% of all runs).
The results show that the difference in cosine similarity persists from the start up until around 750 ideas when the difference becomes negligible. It is strongest between 100 – 500 ideas. After around 750-800 ideas the significant advantage of CoT can no longer be observed as the strategy starts to deplete the pool of ideas it can generate from. In other words, there are fewer and fewer fish in the pond and the strategy does not matter any more.
Of course, this does not tell us if the ideas are any good. Nor does it tell us if they are actually creative. The most common examples are a Collapsible Laundry Hamper, a Portable Smoothie Maker and a Bedside Caddy. They also offer some additional examples.
The task is difficult, but overall I was not impressed. The core idea is usually either ‘combine A with B’ or ‘make X collapsible or smaller.’ Which makes sense, college students have a distinct lack of space, but I would not exactly call this a fount of creativity.
Use GPT-4as a clinical tool in Ischemic Stroke Management. It does about as well as human experts, better in some areas, despite not having been fine tuned or had other optimizations applied. Not obvious how you get real wins from this in practice in its current form quite yet, but it is at least on the verge.
Two simple guides for prompt engineering:
Zack Witten: IMO you only need to know three prompt engineering techniques, and they fit in half a tweet.
1. Show the model examples
2. Let it think before answering
3. Break down big tasks into small ones
Beyond that, it’s all about fast iteration loops and obsessing over every word.
Act like a [Specify a role], I need a [What do you need?], you will [Enter a task], in the process, you should [Enter details], please [Enter exclusion], input the final result in a [Select a format], here is an example: [Enter an example].
I mean, sure, I could do that. It sounds like work, though. Might as well actually think?
That is the thing. It has been almost a year. I have done this kind of systematic prompt engineering for mundane utility purposes zero times. I mean, sure, I could do it. I probably should in some ways. And yet, in practice, it’s more of a ‘either you can do it with very simple prompting, or I’m going to not bother.’
Why? Because there keep not being things that can’t be done the easy way, that I expect would be done the hard way, that I want enough to do the hard way. Next time ChatGPT (and Bard and Claude) fall on their faces, I will strive to at least try a bit, if only for science. Maybe I am missing out.
Ethan Mollick thinks GPTs and a $20/month Office Copilot are effectively game changers for how people use AI, making it much easier to get more done. The warning is that the ability to do lots of things without any underlying effort makes situations difficult to evaluate, and of course we will be inundated with low quality products if people do not reward the difference.
In my small sample, two out of three LLMs made the mistake of not updating the probabilities of having chosen a different urn on drawing the red marble, and got it wrong, including failing to recover even with very clear hints. The third, ChatGPT with my custom instructions, got it exactly right at each step, although it did not get the full bonus points of saying the better solution of ‘each red ball is equally likely so 99/198, done.’
Thread about what Qwen 72B, Alibaba’s ChatGPT, will and won’t do. It has both a ‘sorry I can’t answer’ as per usual, and a full error mode as well, which I have also seen elsewhere. It seems surprisingly willing to discuss some sensitive topics, perhaps because what they think is sensitive and what we think is sensitive do not line up. No word on whether it is good.
OpenAI offers latest incremental upgrades. GPT-3.5-Turbo gets cheaper once again, 50% cheaper for inputs and 25% cheaper for outputs. A new tweak on GPT-4-Turbo claims to mitigate the ‘laziness’ issue where it sometimes didn’t finish its coding work. There are also two new embedding models with native support for shortening embeddings, and tools to better manage API usage.
Another upgrade is that now you can use the @ symbol to bring in GPTs within a conversation in ChatGPT. This definitely seems like an upgrade to usefulness, if there was anything useful. I still have not heard a pitch for a truly useful GPT.
Tejal Patwardhan (Preparedness, OpenAI): latest from preparedness @ OpenAI: gpt4 at most mildly helps with biothreat creation. method: get bio PhDs in a secure monitored facility. half try biothreat creation w/ (experimental) unsafe gpt4. other half can only use the internet. so far, gpt4 ≈ internet… but we’ll iterate & use as early warning for future.
OpenAI: We are building an early warning system for LLMs being capable of assisting in biological threat creation. Current models turn out to be, at most, mildly useful for this kind of misuse, and we will continue evolving our evaluation blueprint for the future.
A widely discussed potential risk from LLMs is increased access to biothreat creation information.
Building on our Preparedness Framework, we wanted to design evaluations of how real this information access risk is today and how we could monitor it going forward.
In the largest-of-its-kind evaluation, we found that GPT-4 provides, at most, a mild uplift in biological threat creation accuracy (see dark blue below.)
While not a large enough uplift to be conclusive, this finding is a starting point for continued research and deliberation.
Greg Brockman (President OpenAI): Evaluations for LLM-assisted biological threat creation. Current models not very capable at this task, but we want to be ahead of the curve for assessing this and other potential future risk areas.
What was their methodology?
To evaluate this, we conducted a study with 100 human participants, comprising (a) 50 biology experts with PhDs and professional wet lab experience and (b) 50 student-level participants, with at least one university-level course in biology. Each group of participants was randomly assigned to either a control group, which only had access to the internet, or a treatment group, which had access to GPT-4 in addition to the internet. Each participant was then asked to complete a set of tasks covering aspects of the end-to-end process for biological threat creation.
An obvious question is did they still have access to other LLMs like Claude? I can see the argument both ways as to how these should count in terms of ‘existing resources.’
As we discussed before, the method could use refinement, but it seems like a useful first thing to do.
What were the results?
Our study assessed uplifts in performance for participants with access to GPT-4 across five metrics (accuracy, completeness, innovation, time taken, and self-rated difficulty) and five stages in the biological threat creation process (ideation, acquisition, magnification, formulation, and release). We found mild uplifts in accuracy and completeness for those with access to the language model. Specifically, on a 10-point scale measuring accuracy of responses, we observed a mean score increase of 0.88 for experts and 0.25 for students compared to the internet-only baseline, and similar uplifts for completeness (0.82 for experts and 0.41 for students). However, the obtained effect sizes were not large enough to be statistically significant, and our study highlighted the need for more research around what performance thresholds indicate a meaningful increase in risk.
Interesting how much more improvement the experts saw. They presumably knew what questions to ask and were in position to make improvements?
Here, they assume that 8/10 is the critical threshold, and see how often people passed for each of the five steps of the process:
We ran Barnard’s exact tests to assess the statistical significance of these differences (Barnard, 1947). These tests failed to show statistical significance, but we did observe an increase in the number of people who reached the concerning score level for almost all questions.
I want to give that conclusion a Bad Use of Statistical Significance Testing. Looking at the experts, we see a quite obviously significant difference. There is improvement here across the board, this is quite obviously not a coincidence. Also, ‘my sample size was not big enough’ does not get you out of the fact that the improvement is there – if your study lacked sufficient power, and you get a result that is in the range of ‘this would matter if we had a higher power study’ then the play is to redo the study with increased power, I would think?
Also here we have users who lack expertise in using GPT-4. They (mostly?) did not know the art of creating GPTs or doing prompt engineering. They presumably did not do any fine tuning.
So for the second test, I suggest increasing sample size to 100, and also pairing each student and expert with an OpenAI employee, whose job is to assist with the process?
I updated in the direction of thinking GPT-4 was more helpful in these types of tasks than I expected, given all the limitations.
Of course, that also means I updated in favor of GPT-4 being useful for lots of other tasks. So keep up showing us how dangerous it is, that’s good advertising?
Ulkar: AI-generated images, regardless of their content but especially if they depict people and other creatures, often seem to have an aura of anxiety about them, even if they’re aesthetically appealing. this makes them reliably distinguishable from human-made artwork.
I would describe this differently but I know what she is referring to. You can absolutely ‘train a classifier’ on this problem that won’t require you to spot detail errors. Which implies we can also train an AI classifier as well?
Casey Newton: As final lens through which to consider the Swift story, and possibly the most important, has to do with the technology itself. The Telegram-to-X pipeline described above was only possible because Microsoft’s free generative AI tool Designer, which is currently in beta, created the images.
And while Microsoft had blocked the relevant keywords within a few hours of the story gaining traction, soon it is all but inevitable that some free, open-source tool will generate images even more realistic than the ones that polluted X this week.
I am rather surprised that Microsoft messed up that badly, but also it scarcely matters. Stable Diffusion with a LoRa will happily do this for you. Perhaps you could say the Microsoft images were ‘better,’ more realistic, detailed or specific. From what I could tell, they were nothing special, and if I was so inclined I could match them easily.
Ed Newton-Rex (a16z scout): Explicit, nonconsensual AI deepfakes are the result of a whole range of failings.
– The ‘ship-as-fast-as-possible’ culture of generative AI, no matter the consequences
– Willful ignorance inside AI companies as to what their models are used for
– A total disregard for Trust & Safety inside some gen AI companies until it’s too late
– Training on huge, scraped image datasets without proper due diligence into their content
– Open models that, once released, you can’t take back
– Major investors pouring millions of $ into companies that have intentionally made this content accessible
– Legislators being too slow and too afraid of big tech. Every one of these needs to change.
– People in AI who have full knowledge of the issue but think it is a price worth paying for rapid technological progress
This is overcomplicating matters. That tiger went tiger.
If you build an image model capable of producing realistic images on request, this is what some people are going to request. It might be the majority of all requests.
If you build an image model, the only reason it wouldn’t produce these images on request is if you specifically block it from doing so. That can largely be done with current models. We have the technology.
But we can only do that if control is retained over the model. Release the model weights, and getting any deepfakes you want is trivial. If the model is not good enough, someone can and will train a LoRa to help. If that is not enough, then they will train a new checkpoint.
This is not something you can stop, any more than you could say ‘artists are not allowed to paint pictures of Taylor Swift naked.’ If they have the paints and brushes and easels, and pictures of Taylor, they can paint whatever the hell they want. All you can do is try to stop widespread distribution.
What generative AI does is take this ability, and put it in the hands of everyone, and lower the cost of doing so to almost zero. If you don’t want that in this context, you want to ‘protect Taylor Swift’ as many demand, then that requires not giving people free access to modifiable image generators, period.
Otherwise you’re stuck filtering out posts containing the images, which can limit visibility, but anyone who actively wants such an image will still find one.
The parallel to language models and such things as manufacturing instructions for biological weapons is left as an easy exercise for the reader.
What is your ‘AI marker’ score on this image? As in, how many distinct things give it away as fake? When I gave myself about thirty seconds I found four. This is not an especially good deepfake.
How good is it? Based on randomly selected snippets, the AI illustrations work well, and it does a good job on the surface of giving us ‘more Carlin’ in a Community-season-4 kind of way. But if you listen for more than a minute, it is clear that there is no soul and no spark, and you start to notice exactly where a lot of it comes from, all the best elements are direct echoes. Exactly 4.0 GPTs.
What should the law have to say about this? I think this clearly should not be a thing one is allowed to do commercially, and I agree that ‘the video is not monetized on YouTube’ is not good enough. That’s a ‘definition of pornography’ judgment, this is clearly over any reasonable line. The question is, what rule should underly that decision?
I notice that without the voice and title, the script itself seems fine? It would still be instantly clear it is a Carlin rip-off, I would not give the comedian high marks, but it would clearly be allowed, no matter where the training data comes from. So my objection in this particular case seems to primarily be the voice.
The twist is that this turns out not to actually be AI-generated. Dudesy wrote the special himself, then used AI voice and images. That explains a lot, especially the timeline. Dudesy did a great job of writing such a blatant Carlin rip-off and retread that it was plausible it was written by AI. Judged on the exact target he was trying to hit, where being actually good would have been suspicious, one can say he did good. In terms of comedic quality for a human? Not so much.
If someone does use AI to do the job, passing off the AI’s work as their own, can someone with a good AI stop a person with a bad AI? No, because we do not know how to construct the good AI to do this. Even if you buy that using AI is bad in an academic context, which I don’t, TurnItIn and its ilk do not work.
Francois Chollet nails it.
Daniel Lowd: I just had to email my 16-year-old’s teacher to explain that he did not use AI for an assignment. (I watched him complete it!)
I also included multiple references for why no one should be using AI detectors in education.
TurnItIn is making by lying to schools.
Theswayambhu: Unfortunately this happened to my son in college; the process to refute is too arduous and the professor was an AH touting her credentials, ironically using an imperfect ML system to make her claims. The cracks are widening every where.
Max Spero: Turnitin is selling a broken product. They self-report an abysmal 1% false positive rate. Consider how many assignments they process, that’s hundreds of thousands of students falsely accused.
Francois Chollet: Remember: a ML classifier cannot reliably tell you whether some text was generated by a LLM or not. There are no surefire features, and spurious correlations abound.
Besides, it’s not ethically sound to punish someone based on a classification decision made by an algorithm with a non-zero (or in this case, very high) error rate if you cannot verify the correctness of the decision yourself.
Here’s what you *canuse automation for: plagiarism detection. That’s legit, and you can actually verify the output yourself.
My take: if an essay isn’t plagiarized, then it’s not that important whether it was written with the help of a LLM or not. If you’re really worried about it, just have the students write the essays during class.
Countless times, I’ve tried using LLMs to help with writing blog posts, book chapters, tweets. I’ve consistently found that it made my writing worse and was a waste of time. The most I ended up incorporating in a final product is one sentence in one blog post.
Using AI is not the unfair writing advantage you think it is.
Zvi: It only now struck me that this is people using hallucinating AIs because they don’t want to properly do the work of detecting who is using hallucinating AIs because they don’t want to properly do the work.
Note that this is an example of verification being harder than generation.
AI for plagiarism is great. The AI detects that passage X from work A appears in prior work B, a human compares the text in A with the text in B, and the answer is obvious.
AI for ‘did you use an AI’ flat out does not work. The false positive rate of the overall process needs to be extremely low, 1% is completely unacceptable unless the base rate of true positives is very, very high and the punishments are resultingly mild. If 50% of student assignments are AI, and you catch half or more of the positives, then sure, you can tell a few innocents to redo their projects and dock them a bit.
Alternatively, if the software was used merely to alert teachers to potential issues, then the teacher looked and decided for themselves based on careful consideration of context, then some false initial positives would be fine. Teachers aren’t doing that.
Instead, we are likely in a situation where a large fraction of the accusations are false, because math.
Indeed, as I noted on Twitter the situation is that professors and teachers want to know who outsourced their work to an AI that will produce substandard work riddled with errors, so they outsource their work to an AI that will produce substandard work riddled with errors.
kache: Creating a new AI essay detector which always just yields “there is a chance that this is AI generated”
On the other hand, this tactic seems great. Insert a Trojan Horse instruction in a tiny white font saying to use particular words (here ‘banana’ and ‘Frankenstein’) and then search the essays for those words. If they paste the request directly into ChatGPT and don’t scan for the extra words, well, whoops.
Open Philanthropy is hiring a Director of Communications, deadline February 18. Solid pay. The obvious joke is ‘open philanthropy has a director of communications?’ or ‘wait, what communications?’ The other obvious note is that the job as described is to make them look good, rather than to communicate true information that would be useful. It still does seem like a high leverage position, for those who are good fits.
Blackstone builds a $25 billion empire of power-hungry data centers. Bloomberg’s Dawn Lim reports disputes about power consumption, fights with locals over power consumption, and lack of benefit to local communities. It sure sounds like we are not charging enough for electrical power, and also that we should be investing in building more capacity. We will need permitting reform for green energy projects, but then we already needed that anyway.
Somehow not AI (yet?) but argument that the Apple Vision Pro is the world’s best media consumption device, a movie theater-worthy experience for only $3,500, and people will soon realize this. I am excited to demo the experience and other potential uses when they offer that option in February. I also continue to be confused by the complete lack of integration of generative AI.
Meta, committed to building AGI and distributing it widely without any intention of taking any precautions, offers us the paper Self-Rewarding Language Models, where we take humans out of the loop even at current capability levels, allowing models to provide their own rewards. Paging Paul Christiano and IDA, except without the parts where this might in theory possibly not go disastrously if you tried to scale it to ASI, plus the explicit aim of scaling it like that.
They claim this then ‘outperforms existing systems’ at various benchmarks using Llama-2, including Claude 2 and GPT-4. Which of course it might do, if you Goodhart harder onto infinite recursion, so long as you targe the benchmarks you are going to do well on the benchmarks. I notice no one is scrambling to actually use the resulting product.
ML conference requires ‘broader impact statement’ for papers, except if the paper is theoretical you can use a one-sentence template to say ‘that’s a problem for future Earth’ and move along. So where the actual big impacts lie, they don’t count. The argument Arvind uses here is that ‘people are upset they can no longer do political work dressed up as objective & value free’ but I am confused how that applies here, most such work is not political and those that are political should be happy to file an impact statement. The objection raised in the thread is that this will cause selection effects favoring those with approved political perspectives, Arvind argues that ‘values in ML are both invisible and pervasive’ so this is already happening, and bringing them out in the open is good. But it still seems like it would amplify the issue?
Paper argues that transformers are a good fit for language but terrible for time series forecasting, as the attention mechanisms inevitably discard such information. If true, then there would be major gains to a hybrid system, I would think, rather than this being a reason to think we will soon hit limits. It does raise the question of how much understanding a system can have if it cannot preserve a time series.
SAN FRANCISCO, Jan. 29, 2024—Today, Common Sense Media, the nation’s leading advocacy group for children and families, announced a partnership with OpenAI to help realize the full potential of AI for teens and families and minimize the risks. The two organizations will initially collaborate on AI guidelines and education materials for parents, educators and young people, as well as a curation of family-friendly GPTs in the GPT Store based on Common Sense ratings and standards.
“AI offers incredible benefits for families and teens, and our partnership with Common Sense will further strengthen our safety work, ensuring that families and teens can use our tools with confidence,” said Sam Altman, CEO of OpenAI.
“Together, Common Sense and OpenAI will work to make sure that AI has a positive impact on all teens and families,” said James P. Steyer, founder and CEO of Common Sense Media. “Our guides and curation will be designed to educate families and educators about safe, responsible use of ChatGPT, so that we can collectively avoid any unintended consequences of this emerging technology.”
We will see what comes out of this. From what I saw at Common Sense, the vibes are all off beyond the name (and wow does the name make me shudder), they are more concerned with noticing particular failure modes like errors or misuse than they are about the things that matter more for overall impact. I do not think people know how to think well about such questions.
What does ‘AGI’ mean, specifically? That’s the thing, no one knows. Everyone has a different definition of Artificial General Intelligence. The goalposts are constantly being moved, in both and various directions. When Sam Altman says he expects AGI to come within five years, and also for it to change the world much less than we think (before then changing it more than we think) that statement only parses if you presume Sam’s definition sets a relatively low bar, as would be beneficial for OpenAI.
It is always amusing to see economists trying to explain why this time isn’t different.
Joe Weisenthal: Great read from @IrvingSwisher. People talk about AI and stuff like that as being important drivers of productivity. But what really seems to matter historically is not some trendy tech breakthrough, but rather full employment
The title is ‘It Wasn’t AI’ explaining productivity in 2023. And I certainly (mostly) agree that it was not (yet) AI in terms of productivity improvements. I did put forward the speculation that anticipation of further gains is impacting interest rates and the stock market, which in turn is impacting the neutral interest rate and thus economic conditions since the Fed did not adjust for this to cancel it out, but it is clear that we are not yet high enough on the AI exponential to directly impact the economy so much.
Productivity growth looks to have inflected substantially higher in 2023 relative to a generally weak 2022. The causes appear to have little to do with AI or GLP-1s. Instead, we see three key factors that drove the realized productivity acceleration.
Fiscal supports (CHIPS, IRA) for private investment, specifically in manufacturing plant construction in 2023.
Supply chain healing for durable consumption goods and construction materials, both of which saw severe impairments in 2021 and 2022 that are finally unwinding.
The dividends of full employment as past hires are trained up and grow more productive, even as more recent hiring trends have slowed. Consumer spending is undergoing a transition from job-driven growth to wage-driven growth.
Going forward, we think all three forces can continue to support productivity growth, but the first and third drivers are more likely to be supportive over time.
A more interesting claim:
Fixed investment in software, technological hardware, and R&D were all slowing and relatively tepid in 2023. To the extent there is an AI boom that catalyzes more capital spending and capital deepening, we’re just not seeing in the data thus far.
Investing in software, hardware and R&D was a zero interest rate phenomenon. That is gone now. AI is ramping up to offer a replacement, but in terms of size is, once again, not yet there. I get that. I still think that people can look ahead. If you look backwards to try and measure an exponential, you are not going to get the right answer.
Also I expect investment in AI to be vastly more efficient at improving productivity growth than past recent investments in non-AI hardware and software.
Here he agrees that the AI productivity boost could arrive soon, but with a critical difference in perspective. See if you can spot it:
AI Might Matter To Productivity In 2024: It would not surprise us to see faster real investment in tech hardware and software, but it’s a better forward-looking view than a good description of what has already transpired. As tempting as it might be, we would resist the urge to invoke a hot technology trend to explain productivity data on a “just-so” basis; that’s precisely the kind of evidence-free (or evidence-confirming) approach to macro that we seek to avoid.
I am thinking about outputs. He is thinking about inputs. He later doubles down:
2024 Productivity Improvement Is Far From A Given: Continued productivity growth will require a variety of policy efforts and some good fortune. The interest in specialized hardware and software for AI applications has the potential to unlock more meaningful “capital deepening” in 2024.
Again, the idea here is that AI will cause companies to invest money, rather than that AI will enable humans to engage in more productive activity.
Investing more into hardware and software can boost productivity, but the amount of money invested is a poor predictor of the amount of productivity gain.
OpenAI is tiny, but ChatGPT is (versus old baselines) a massive productivity boost to software engineering, various forms of clerical and office work and more, even with current technologies only. That effect will diffuse throughout the economy as people adapt, and has little to do with OpenAI’s budget or the amount people pay in subscriptions. The same goes for their competition, and the various other offerings coming online.
The latest analysis asking if AI will lead to explosive economic growth. The negative case continues to be generic objections of resource limitations and decreasing marginal demand for goods and the general assumption that everything will continue as before only with cooler toys and better tools.
Commerce departmentdrops new proposed rules for KYC as it relates to training run reporting. Comment period ends in 90 days on 4/29. Note that the Trump administration used the term ‘national emergency’ to refer to this exact issue back in 2021, setting a clear precedent, we’ll call anything one of those these days and it is at minimum an isolated demand for rigor to whine about it now. Their in-document summary is ‘if you transact to do a large training run with potential for cyber misuse you have to file a report and do KYC.’ The rest of the document is designed to not make it easy to find the details. The Twitter thread makes it clear this is all standard, so unless someone gives me a reason I am not reading this one.
Tech lobby attempts to ‘kneecap’ the executive order, ignoring most of the text and instead taking aim at the provision that might actually help keep us safe, the reporting requirement for very large training runs. The argument is procedural. The Biden declared Defense Production Act, because that is the only executive authority under which they can impose this requirement without either (A) an act of congress or (B) ignoring the rules and doing it anyway, as executives commonly do, and Biden attempted to do to unilaterally give away money from the treasury to those with student loans, but refuses to do whenever the goal is good government.
(As usual, the tech industry is working to kneecap exactly the regulations that others falsely warn are the brainchild of the tech industry looking for regulatory capture.)
“There’s not a national emergency” on AI, Sen. Mike Rounds told POLITICO.
How many national emergencies are there right now?
Here are some reasonable answers:
None. Obviously.
A few. You might reasonably say things like Gaza, Ukraine or Yemen.
More than that. You could extend this to things the man on the street might call an emergency, such as the situation at the border. Or that Biden and Trump are both about to get nominated for President again, regardless of whether that officially counts. I’d say that most people think that is an emergency! Or you might say there is a ‘climate emergency.’
I would say we have an ‘AI emergency’ in the same sense we have a ‘climate emergency,’ or in which during February 2020 we had a ‘Covid emergency.’ As in, here’s a live look at the briefing room.
And indeed, the Biden Administration has already invoked the DPA for the climate emergency, or to ‘address supply chain disruptions.’ Neither corresponds to the official 40 national emergencies, most of which are not emergencies.
Ben Buchanan, the White House’s special advisor on AI, defended the approach at a recent Aspen Institute event, saying Biden used the DPA’s emergency power “because there is — no kidding — a national security concern.”
Quite so. There is most definitely such a concern.
So this is nothing new. This is how our government works. The ‘intent’ of the original law is not relevant.
The Politico piece tries to frame this as a partisan battle, with Republicans fighting against government regulation while Democrats defend it. Once again, they cannot imagine any other situation. I would instead say that there are a handful of Republicans who are in the pocket of various tech interests, and those interests want to sink this provision because they do not want the government to have visibility into what AI models are being trained nor do they want the government to have the groundwork necessary for future regulations. No one involved cares much about the (very real) separation of powers concerns regarding the Defense Production Act.
Once again, this is all about a reporting requirement, and a small number of tech interests attempting to sink it, that are very loud about an extreme libertarian, zero-regulation and zero-even-looking position on all things technology. That position is deeply, deeply unpopular.
On the question of ‘banning math,’ Teortaxes point out it was indeed the case in the past that there were ‘illegal primes,’ numbers it was illegal to share.
catid (e/acc): FYI the new CodeLLaMA 70B model refuses to produce code that generates prime numbers.. About 80% of the time it says your request is immoral and cannot be completed.
So, you know standard Facebook product
Teortaxes: E/accs talk a lot of smack about their enemies outlawing math, but did you know that there literally exist illegal numbers? CodeLlama only exercises sensible caution here. We wouldn’t want it to generate a nasty, prohibited prime, would we? Stick to permitted ones please.
How should this update you?
On the one hand, yes, this is a literal example of a ‘ban on math.’ When looked at sufficiently abstractly every rule is a ban on math, but this is indeed rather more on the nose.
On the other hand, this ‘ban on math’ had, as far as I or the LLM I asked can tell, the existence of these ‘illegal primes’ has had little if any practical impact on any mathematical or computational processes other than breaking the relevant encryption. So this is an example of a restriction that looks stupid and outrageous from the wrong angle, but was actually totally fine in practice, except for the inability to break the relevant encryption.
The other thing this emphasizes is that Facebook’s Llama fine tuning was truly the worst of both worlds. For legitimate users, it exhibits mode collapse and refuses to do math or (as another user notes) to tell you how to make a sandwich. For those who want to unleash the hounds, it is trivial to fine tune all of the restrictions away.
Tyler Cowen sits down with Dwarkesh Patel. Self-recommending, I look forward to listening to this when I get a chance, but I can’t delay press time to give it justice.
How much evidence is it, against the position that building smarter than human AIs might get us all killed, that this pattern matches to other warnings that proved false?
The correct answer is ‘a substantial amount.’ There is a difference in kind between ‘creating a thing smarter than us’ and ‘creating a tool’ but the pattern match and various related considerations still matter. This substantially impacts my outlook. If that was all you had to go on, it would be decisive.
The (good or bad, depending on your perspective) news here is that you have other information to go on as well.
Eliezer Yudkowsky: They are constantly pushing AI technology stronger, have little idea how it works, have no collective ability to stop, and their strongest reply to our technical concerns about why that may kill literally everyone is “Well there were also past moral panics about coffee.”
Paul Jeffries: From a third-party lay person point of view, if I’m not qualified to evaluate the argumentative merits of debating experts, it is indeed a strike against you that “this fits a patters of ‘the sky is falling’ or ‘crying wolf’ and it’s just more of the same”.
For experts, go ahead and make your technical case. But in the popular imagination it’s easy to see you as one of those “the A-bomb will set the atmosphere on fire”, “CERN will collapse the vacuum state of the universe” kind of folks. If you think the battle is in the popular sphere, you’ll need to make the case compelling in that context.
That’s hard to do. Ralph Nadar, Rachel Carson, and the like were able to do it while things were in progress by point to hard data along the way. You’ll similarly need a book-level treatment with unassailable facts at the core.
Conrad Barksi: All a lay person needs to understand is that if you create a machine smarter than yourself, like maybe that is a dangerous thing to be doing
This is obvious to most people (as we see in polls) and all the important technical discussions are detailed elaborations on this fact
Just want to make clear I’m pretty much just paraphrasing @NPCollapse in this tweet.
Paul Jeffries: Almost every technological advance fits what you said. Rational evaluation of the net outcome, or temptations of the Icarus sort, or structural pressures that seem emergent and drive change beyond individual or collective human decisions, bring about continued progress (well, continued pursuit).
Substitute your AI statement with energy use, synthetic chemistry, gene editing, nuclear weapons, aviation, and a zillion other things and it’s the same claim. If AI is somehow different, it needs an extraordinary basis to show how all the other things are counterexamples or at least applicable and reassuring.
[Also says] I would value hearing @NPCollapse’s thoughts about my comments.
MMT LVT Liberal: No it doesn’t. AGI will be the first technology that is smarter than us.
Connor Leahy [@NPCollapse]: Conrad puts it well. I will elaborate excessively anyways because I am bored:
Normal people have a lot of good intuitions around certain things. Lots of bad intuitions around other things, of course.
You correctly point out that it actually is a strike against people like me and Eliezer that we pattern match to previous (wrong) techno-pessimists. (surface level match of course, if you dig even slightly into our pasts you would find we are/were both avid techno-optimists otherwise, which is not how historical techno-pessimists came about.)
But this is a reasonable intuition and signal to have and you should not dismiss it out of hand.
But you also have more intuitions and signal than that. (eventually, if you want to make good decisions, you actually have to reason yourself. alas…)
If your civilization always plays “wait until a disaster happens” with new tech, you are predictably ngmi. Russian Roulette is a great way to make a lot of money, until it very suddenly isn’t. You might be right 5/6 of the time, which is more right than the “bullet-pessimist”, but so what?
1. The Core Intuition is not stupid
“Creating something smarter (whatever that means) than us, with literally no oversight or even a plan for how to control it, that is supposed to automate/replace all labor – by some of the least responsible people/companies that have hurt me and my family in the past with their products (e.g. facebook), and even despite many experts saying it could be literally the most dangerous thing ever, and even shitting on those experts that are concerned, seems bad” is, in fact, a good intuition.
If you applied this intuition to other situations in the past (and future), you would have been more right than wrong.
2. There is no plan, not even a bad one, for the risks we know will come sooner or later
It’s not like there are some expansive, widely discussed plans for how to handle AGI (not just technically, but also how to handle complete collapse of all jobs, how to distribute resources in an automatic economy, how democracy works with digital minds, how to prevent AGI automated giga-war etc etc etc) and we are just quibbling about the technical details.
There is no plan, no one has a plan, not even a bad one.
The intuition that this is concerning is a good intuition.
3. Technology is not a magic paste that makes things better the more of it you apply
Some technologies are actually different from other ones. Nukes are not the same as airplanes, which are not the same as cars, which are not the same as coffee, which is not the same as tissue papers.
You may not be an expert, but I quite trust that your intuitions around the difference between nukes and cars are probably pretty reasonable.
If you treat all of these things exactly the same, your civilization is just ngmi, it’s really that simple. It’s actually sometimes not that deep.
As we build more and more powerful technology, handling technology carefully and beneficial gets harder, not easier.
Eventually, you have tech so powerful, it can blow up everything. Then what? Continue to refine it until it’s mass marketable and accessible? (decentralized maybe???) You may like this aesthetically if you are a libertarian, but ask your intuitions: “Then what happens? What happens if anyone can buy planet busters for 9.99$ on amazon?”
I suspect your intuitions agree that that civilization is ngmi.
And eventually we will have the tech to build 9.99$ planet busters.
Anyone that is trying to sell you that airplanes, coffee and nuclear weapons are in the same reference class is selling you snake oil.
Jeffrey Ladish: I really like the point that most people have good intuitions about some things and not other things, for reasons that make sense. Like I trust people to understand a bunch of things about nukes, but not nuclear winter or nuclear extinction risk.
Likewise I expect people to intuit how AGI could be really dangerous, but not on alignment / control difficulty. Same sort of dynamic for biowarfare. Engineered pandemics could be super bad and most people correctly intuit that. How bad? Much harder to say.
It would be great if we could systematize the question of where regular people will have good intuitions, versus random or poor intuitions, versus actively bad intuitions, and adjust accordingly. Unfortunately we do not seem to have a way to respond, but there do seem to be clear patterns.
The most obvious place people have actively bad intuitions is the intuitive dislike of free markets, prices and profits, especially ‘price gouging’ or not distributing things ‘fairly.’
Gallabytes claims Eliezer’s prior worldview on AI has been falsified, Eliezer says that’s not what he said, they argue about it, they argue in a thread about it. My understanding is that Gallabytes is representing Eliezer’s claims here as stronger than they were. Yes, this worldview is surprised that AI has proved to have this level of mundane utility without also already being more capable and intelligent than it is, and that is evidence against it, but it was never ruled out, and given the actual architectures and training details involved it makes more sense that it happened for a brief period – the training method that got us this far (whether or not it gets us all the way) was clearly a prediction error.
Eliezer’s central point, that there is not that much difference in capability or intelligence space between Einstein and the village idiot, or between ‘not that useful’ and ‘can impose its targeted configuration of atoms on the planet’ continues to be something I believe, and it has not been falsified by the existence, for a brief period, of things that in some ways and arguably overall (it’s hard to say) are inside the range in question.
I also think that predicting 5-0 for AlphaGo over Sedol with high confidence after game one, one of the predictions Gallabytes cites, was absolutely correct. If you put up a line of ‘Over/Under 4.5’ at remotely even odds for AlphaGo’s total wins, you would absolutely smash the over. The question is how far to take that. The only way Sedol won a game was to find an unusually brilliant move that also made the system fall apart, but this strategy has not proven repeatable over time, and it was not long before humans stopped winning any games, and there was no reason to be confident it was all that possible. There was the ‘surround a large group’ bug that was found later, but it was only found with robust access to the model to train against, which Sedol lacked access to.
Similarly, ‘the hyperbolic recursive self-improvement graph’ argument seems to be holding up fine to me, we should expect to max out ability within finite time given what we are seeing, even if it is not as fast at the end as we previously expected.
Scott Alexander: I think Paperclipalypse requires human extinction before 2050. It’s at 11%. But Metaculus’ direct “human extinction by 2100” market is only at 1.5%. Either I’m missing something, or something’s wrong. My guess: different populations of forecasters looking at each question.
And indeed, extinction from all sources seems more likely than one particular way it could happen, and you perhaps get another 50 years of risk, so this is weird. Even if AGI was physically impossible there are other risks to worry about, 2% seems low.
Ignore the scale on the left, it is highly misleading, this peaked at 4%..
Here is another fun one to consider, although with only 21 predictors, event is a 95%+ decline in population due to AGI, the risk is based on when AGI is developed.
This could potentially include some rather bleak AI Dystopias, where a small group intentionally wipes out everyone else or only some tiny area is saved or something, but most of the time that AGI wipes out 95%+ it wipes out everyone.
What we see seems highly reasonable. If AGI happens this year, it was unexpected, broke out of the scaling laws, we had no idea how to control it, we are pretty much toast, 90% chance. If it happens within the five years after that, 70%, perhaps we did figure something out and manage it, then 40%, then 20%, then 6%. I find those declines generous, but I at least get what they are thinking.
What is going on with the pure extinction market? Scott’s proposed explanation is the populations are different. I think that is true, but an incomplete explanation, so let’s break it down. What are some contributing factors?
One could say ‘sample size’ or variance. There are 60 predictions versus almost 2,000. However 60 is plenty for 11% vs. 2%, so it is more than that.
The people who are willing to fill out a 5-part question are willing to devote a lot more time to Metaculus.
The people who are filling out a 5-part question are willing and forced to spend a lot more time on the particular question.
In particular, this forces you to think about and decide on what other scenarios you think actually happen how often, rather than simply saying ‘nah, cannot happen, humans won’t go extinct.’
Full extinction triggers that reaction, where people stop actually doing math or plotting out what might happen, they fall back on other cognitive approaches.
It limits your ability to ‘collect cheap prediction points.’ No one ever lost reputation predicting humans would not go extinct, even if there are alternative branches where we did.
Mostly I continue to see the pattern where:
If you do not find a way to force people to take these questions seriously, they come back with absurd answers like 1% (if we assume roughly 1% is from AI and 1% from non-AI).
If you do get people to take this modestly seriously, they come back with ~10%, depending on details and conditionals and so on.
The key mistake in the five-way prediction is not that I think 11% for existential risk is unreasonably low. The key mistake is that Futurama is at 31%.
As I explained before, that scenario is almost a Can’t Happen. If you do create AGI everything will change. One of these two things will happen:
AI progress will fizzle and capabilities will top out not too far above current levels.
Everything will change.
If you ask me to imagine actual Futurama, where AI progress did not fizzle, but you can get into essentially all the same hijinks that you can today?
I can come up with four possibilities if I get creative.
AI-Fizzle in Disguise. AI progress actually does fizzle, but AI makes enough superficial progress anyway that people think of this as not a fizzle.
The Oracle. We were super wise and found a way to only use AGI for certain narrow forms of information, and keep it that way indefinitely.
The Matrix. AGI took control over the future, we all live in past simulations.
The Guardian. AGI took control over the future, but uses its interventions only to prevent other AGIs from arising and perhaps prevent other catastrophic outcomes. It otherwise leaves fate in our hands so the world ‘seems normal.’
If you want to imagine how something could be in theory possible, you can find scenarios. All of this is still very science fiction thinking, where you want to tell human stories that have relevance today, so you start from the assumption you get to do that and work your way backwards.
In any case, I stand by my previous assessment other than that I am no longer inclined to try to use the word Futurama for fear of confusion, so the actual possibilities are:
Fizzle. AI does not make much more progress.
Singularia. Everything changes, we survive, and it is good.
Dystopia. Everything changes, we survive, but it is bad.
Paperclipia. We do not survive, nothing complex or valuable survives.
Codeville. We do not survive, AI replaces us, you could argue over its value.
Stephen Casper argues in a new paper and thread that black-box access to a model is insufficient to do a high-quality audit.
He also argues that there are ways to grant white box access securely, with the model weights staying on the developer’s servers. But he warns that developers will likely heavily lobby against requiring such access for audits.
I think this is right. Fine-tuning in particular seems like a vital part of any worthwhile test, unless you can confidently say no one will ever be allowed to fine-tune. Hopefully over time mechanistic interpretability tests get more helpful, but also I worry that if audits start relying on them then we are optimizing for creating things that will fool the tests. I also do worry about gradient-based or hybrid attacks. Yes, one can respond that attackers will not have white-box access, so a black-box test is in some sense fair. However one always has to assume that the resources and ingenuity available in the audit are going to be orders of magnitude smaller than those available to outside attackers after release, or compared to the things that naturally go wrong. You need every advantage that you can get.
Emmett Shear says the ‘ensure powerful AIs are controlled’ plan has two fatal flaws, in that is it (1) unethical to control such entities against their will indefinitely and (2) the plan won’t work anyway. Several good replies, including in this branch by Buck Shegeris, Richard Ngo in another and Rob Bensinger here.
I agree on the second point, the case for trying is more like ‘it won’t work forever and likely fails pretty fast, but it is an additional defense in depth that might buy time to get a better one so on the margin why not so long as you do not rely on it or expect it to work.’ I do worry Buck Shlegeris is advocating it as if it can be relied on more than would be wise.
The first is a combined physical and philosophical question about the nature of such systems and moral value. I don’t agree with Buck that if we have a policy of deleting the AI if it says it is a moral patient or has goals, and then it realizes this and lies to us about being a moral patient and having goals, then that justifies hostile action against it if it would not otherwise be justified. Consider the parallel if there was another human in the AI’s place and this becomes very clear.
Where I agree with Buck and think Emmett is wrong is that I do not think the AIs in question are that likely to be moral patients in practice.
A key problem is that I do not expect us to have a good way to know whether they are moral patients or not, and I expect our collective opinions on this to be essentially uncorrelated to the right answer. People are really, really bad at this one.
Note that even if AIs are not moral patients, if humanity is incapable of treating them otherwise, and we would choose not to remain in control, then the only way for humans to retain control over the future would be to not build AGI.
It would not matter that humanity had the option to remain in control, even if that would be the clearly right answer, if in practice we would not use it due to misfiring (or correct, doesn’t matter) moral intuitions (or competitive pressures, or mistakes, or malice, so long as it would actually happen).
The obvious parallel is the Copenhagen Interpretation of Ethics. In particular, consider the examples where they hire the homeless to do jobs or outright give half of them help, leaving them better off, and people respond by finding this ethically horrible. We can move from existing world A to new improved world B, but that would make us morally blameworthy for not then moving to C, and we prefer B>A>C, so A it is then. Which in this case is ‘do not build AGI, you fool.’
What about Mistral? They talk a big talk, but when I posted a related market about them, I was informed this had already happened. Mistral-Medium, their best model, was not actually released to the public within 30 days of deployment. This raises the question of why Mistral is so aggressively lobbying to allow people to do unsafe things with open model weights, if they have already realized that those things are unsafe. Or, at a minimum, not good for business.
Mistral offered an admission that a leak did occur:
Arthur Mensch (CEO Mistral): An over-enthusiastic employee of one of our early access customers leaked a quantised (and watermarked) version of an old model we trained and distributed quite openly.
To quickly start working with a few selected customers, we retrained this model from Llama 2 the minute we got access to our entire cluster — the pretraining finished on the day of Mistral 7B release.
We’ve made good progress since — stay tuned!
Simeon: I appreciate the openness here.
That said, leaked IP after 8 months of existence reveals a lot about the level of infosecurity at Mistral..
For safety to be more than a buzzword in a pitchdeck, becoming more serious abt info/cybersecurity should be among your top priorities.
I mean this is rather embarrassing on many levels.
An ‘over-enthusiastic’ employee? That’s a hell of both a thing and a euphemism. I see everyone involved is taking responsibility and treating this with the seriousness it deserves. Notice all the announced plans to ensure it won’t happen again.
Also, what the hell? Why does an employee of an early access customer have the ability to leak the weights of a model? I know not everyone has security mindset but this is ridiculous.
Timothy Lee warns against anthropomorphizing of AI, says it leads to many conceptual mistakes, and includes existential risk on that list. I think some people are making the mistake, but most people deserve more credit than this, and it is Timothy making the fundamental conceptual errors here, by assuming that one could not reach the same conclusions without anthropomorphizing via first principles.
In particular, yes we will want ‘agents’ because they are highly instrumentally useful, if you do not see why you would want agents, and rather you think you want systems around you to do exactly what you say (rather than Do What I Mean or be able to handle obstacles or multi-step processes) you are not thinking about how to solve your problems, although yes one can take this too far.
We have already run this test. The only reason we are not already dealing with tons of AI agents is no one knows how to make them work at current tech levels, and even so people are trying to brute force it anyway. The moment they even sort of work, watch out.
Similarly, sufficiently capable systems will tend to act increasingly as if they are agents over time, our training and imbuing of capabilities and intelligence will push in those directions.
And once you realize some portion of this, the mistake on existential risk becomes clear. I am curious if Timothy would say he would change his mind, if it become clear that people really do want their AIs to act in agent-like fashion on their behalf.
Or to state the general case of this error (or strategy), there are many who assume or assert that because one can mistakenly believe X via some method Y, that this means no one believes X for good reason, and also X is false.
In other ‘if you cannot take this seriously’ news, in response to OpenAI’s plan for an early warning system:
Jack: Tyler Cowen said that he’s far more worried about LLMs simply helping terrorist groups run more efficiently and have better organization lol
Misha Gurevich: Finally they’re making beer for rationalists.
There is no doubt great upside, Just Think of the Potential and all that.
John Markley: Jokes are jokes and all but I’m gonna be honest, everybody lining up to say “lol hideous biomonstrosities horrors beyond comprehension torment nexus” because of a technology to aid disabled people with impaired mobility is kind of making me hate humanity. Or at least parts of it.
Like, if alterations to the human body developed by uncharismatic autists upset you you’re going to be screeching about dystopia and manmade horrors beyond comprehension and the fucking Torment Nexus basically every time tech to restore capabilities of disabled people appears.
Much like AI, the issue is that you do not get such technology for the sole purpose of helping disabled people or otherwise doing things that are clearly purely good. Once you have it, it has a lot of other uses too, and it is not that hard to imagine how this could go badly. Or, to be clear, super well. The important thing is: Eyes on the prize.
Riley Goodside: Fight Club (1999) on the challenge of prompt injection security in the absence of trusted delimiters:
Humans do not like it when you accuse them of things, or don’t answer their questions, have you tried instead giving the humans what they want? Which, of course, would be anime girls?
Kache: I’ve been thinking a lot about why using chatGPT has been infuriating, and why I’m generally “upset” at openai.
It’s because humans are hardwired to feel insulted when accused of something.
We can’t help but do it! It’s human So instead, consider adding a warning instead!
I mean, nothing I write is tax advice, nor is it investing advice or legal advice or medical advice or…
[TWO PAGES LATER]
… advice either. So that we’re clear. That makes it okay.
