revenge porn

nudify-app’s-plan-to-dominate-deepfake-porn-hinges-on-reddit,-docs-show

Nudify app’s plan to dominate deepfake porn hinges on Reddit, docs show


Report: Clothoff ignored California’s lawsuit while buying up 10 rivals.

Clothoff—one of the leading apps used to quickly and cheaply make fake nudes from images of real people—reportedly is planning a global expansion to continue dominating deepfake porn online.

Also known as a nudify app, Clothoff has resisted attempts to unmask and confront its operators. Last August, the app was among those that San Francisco’s city attorney, David Chiu, sued in hopes of forcing a shutdown. But recently, a whistleblower—who had “access to internal company information” as a former Clothoff employee—told the investigative outlet Der Spiegel that the app’s operators “seem unimpressed by the lawsuit” and instead of worrying about shutting down have “bought up an entire network of nudify apps.”

Der Spiegel found evidence that Clothoff today owns at least 10 other nudify services, attracting “monthly views ranging between hundreds of thousands to several million.” The outlet granted the whistleblower anonymity to discuss the expansion plans, which the whistleblower claimed was motivated by Clothoff employees growing “cynical” and “obsessed with money” over time as the app—which once felt like an “exciting startup”—gained momentum. Because generating convincing fake nudes can cost just a few bucks, chasing profits seemingly relies on attracting as many repeat users to as many destinations as possible.

Currently, Clothoff runs on an annual budget of around $3.5 million, the whistleblower told Der Spiegel. It has shifted its marketing methods since its launch, apparently now largely relying on Telegram bots and X channels to target ads at young men likely to use their apps.

Der Spiegel’s report documents Clothoff’s “large-scale marketing plan” to expand into the German market, as revealed by the whistleblower. The alleged campaign hinges on producing “naked images of well-known influencers, singers, and actresses,” seeking to entice ad clicks with the tagline “you choose who you want to undress.”

A few of the stars named in the plan confirmed to Der Spiegel that they never agreed to this use of their likenesses, with some of their representatives suggesting that they would pursue legal action if the campaign is ever launched.

However, even celebrities like Taylor Swift have struggled to combat deepfake nudes spreading online, while tools like Clothoff are increasingly used to torment young girls in middle and high school.

Similar celebrity campaigns are planned for other markets, Der Spiegel reported, including British, French, and Spanish markets. And Clothoff has notably already become a go-to tool in the US, not only targeted in the San Francisco city attorney’s lawsuit, but also in a complaint raised by a high schooler in New Jersey suing a boy who used Clothoff to nudify one of her Instagram photos taken when she was 14 years old, then shared it with other boys on Snapchat.

Clothoff is seemingly hoping to entice more young boys worldwide to use its apps for such purposes. The whistleblower told Der Spiegel that most of Clothoff’s marketing budget goes toward “advertising posts in special Telegram channels, in sex subs on Reddit, and on 4chan.”

In ads, the app planned to specifically target “men between 16 and 35” who like benign stuff like “memes” and “video games,” as well as more toxic stuff like “right-wing extremist ideas,” “misogyny,” and “Andrew Tate,” an influencer criticized for promoting misogynistic views to teen boys.

Chiu was hoping to defend young women increasingly targeted in fake nudes by shutting down Clothoff, along with several other nudify apps targeted in his lawsuit. But so far, while Chiu has reached a settlement shutting down two websites, porngen.art and undresser.ai, attempts to serve Clothoff through available legal channels have not been successful, deputy press secretary for Chiu’s office, Alex Barrett-Shorter, told Ars.

Meanwhile, Clothoff continues to evolve, recently marketing a feature that Clothoff claims attracted more than a million users eager to make explicit videos out of a single picture.

Clothoff denies it plans to use influencers

Der Spiegel’s efforts to unmask the operators of Clothoff led the outlet to Eastern Europe, after reporters stumbled upon a “database accidentally left open on the Internet” that seemingly exposed “four central people behind the website.”

This was “consistent,” Der Spiegel said, with a whistleblower claim that all Clothoff employees “work in countries that used to belong to the Soviet Union.” Additionally, Der Spiegel noted that all Clothoff internal communications it reviewed were written in Russian, and the site’s email service is based in Russia.

A person claiming to be a Clothoff spokesperson named Elias denied knowing any of the four individuals flagged in their investigation, Der Spiegel reported, and disputed the $3 million budget figure. Elias claimed a nondisclosure agreement prevented him from discussing Clothoff’s team any further. However, soon after reaching out, Der Spiegel noted that Clothoff took down the database, which had a name that translated to “my babe.”

Regarding the shared marketing plan for global expansion, Elias denied that Clothoff intended to use celebrity influencers, saying that “Clothoff forbids the use of photos of people without their consent.”

He also denied that Clothoff could be used to nudify images of minors; however, one Clothoff user who spoke to Der Spiegel on the condition of anonymity, confirmed that his attempt to generate a fake nude of a US singer failed initially because she “looked like she might be underage.” But his second attempt a few days later successfully generated the fake nude with no problem. That suggests Clothoff’s age detection may not work perfectly.

As Clothoff’s growth appears unstoppable, the user explained to Der Spiegel why he doesn’t feel that conflicted about using the app to generate fake nudes of a famous singer.

“There are enough pictures of her on the Internet as it is,” the user reasoned.

However, that user draws the line at generating fake nudes of private individuals, insisting, “If I ever learned of someone producing such photos of my daughter, I would be horrified.”

For young boys who appear flippant about creating fake nude images of their classmates, the consequences have ranged from suspensions to juvenile criminal charges, and for some, there could be other costs. In the lawsuit where the high schooler is attempting to sue a boy who used Clothoff to bully her, there’s currently resistance from boys who participated in group chats to share what evidence they have on their phones. If she wins her fight, she’s asking for $150,000 in damages per image shared, so sharing chat logs could potentially increase the price tag.

Since she and the San Francisco city attorney each filed their lawsuits, the Take It Down Act has passed. That law makes it easier to force platforms to remove AI-generated fake nudes. But experts expect the law will face legal challenges over censorship fears, so the very limited legal tool might not withstand scrutiny.

Either way, the Take It Down Act is a safeguard that came too late for the earliest victims of nudify apps in the US, only some of whom are turning to courts seeking justice due to largely opaque laws that made it unclear if generating a fake nude was illegal.

“Jane Doe is one of many girls and women who have been and will continue to be exploited, abused, and victimized by non-consensual pornography generated through artificial intelligence,” the high schooler’s complaint noted. “Despite already being victimized by Defendant’s actions, Jane Doe has been forced to bring this action to protect herself and her rights because the governmental institutions that are supposed to protect women and children from being violated and exploited by the use of AI to generate child pornography and nonconsensual nude images failed to do so.”

Photo of Ashley Belanger

Ashley is a senior policy reporter for Ars Technica, dedicated to tracking social impacts of emerging policies and new technologies. She is a Chicago-based journalist with 20 years of experience.

Nudify app’s plan to dominate deepfake porn hinges on Reddit, docs show Read More »

trump-to-sign-law-forcing-platforms-to-remove-revenge-porn-in-48-hours

Trump to sign law forcing platforms to remove revenge porn in 48 hours

Likely wearisome for victims, the law won’t be widely enforced for about a year, while any revenge porn already online continues spreading. Perhaps most frustrating, once the law kicks in, victims will still need to police their own revenge porn online. And the 48-hour window leaves time for content to be downloaded and reposted, leaving them vulnerable on any unmonitored platforms.

Some victims are already tired of fighting this fight. Last July, when Google started downranking deepfake porn apps to make AI-generated NCII less discoverable, one deepfake victim, Sabrina Javellana, told The New York Times that she spent months reporting harmful content on various platforms online. And that didn’t stop the fake images from spreading. Joe Morelle, a Democratic US representative who has talked to victims of deepfake porn and sponsored laws to help them, agreed that “these images live forever.”

“It just never ends,” Javellana said. “I just have to accept it.”

Andrea Powell—director of Alecto AI, an app founded by a revenge porn survivor that helps victims remove NCII online—warned on a 2024 panel that Ars attended that requiring victims to track down “their own imagery [and submit] multiple claims across different platforms [increases] their sense of isolation, shame, and fear.”

While the Take It Down Act seems flawed, passing a federal law imposing penalties for allowing deepfake porn posts could serve as a deterrent for bad actors or possibly spark a culture shift by making it clear that posting AI-generated NCII is harmful.

Victims have long suggested that consistency is key to keeping revenge porn offline, and the Take It Down Act certainly offers that, creating a moderately delayed delete button on every major platform.

Although it seems clear that the Take It Down Act will surely make it easier than ever to report NCII, whether the law will effectively reduce the spread of NCII online is an unknown and will likely hinge on the 48-hour timeline overcoming criticisms.

Trump to sign law forcing platforms to remove revenge porn in 48 hours Read More »

trump’s-hasty-take-it-down-act-has-“gaping-flaws”-that-threaten-encryption

Trump’s hasty Take It Down Act has “gaping flaws” that threaten encryption


Legal challenges will likely immediately follow law’s passage, experts said.

Everyone expects that the Take It Down Act—which requires platforms to remove both real and artificial intelligence-generated non-consensual intimate imagery (NCII) within 48 hours of victims’ reports—will likely pass a vote in the House of Representatives tonight.

After that, it goes to Donald Trump’s desk, where the president has confirmed that he will promptly sign it into law, joining first lady Melania Trump in strongly campaigning for its swift passing. Victims-turned-advocates, many of them children, similarly pushed lawmakers to take urgent action to protect a growing number of victims from the increasing risks of being repeatedly targeted in fake sexualized images or revenge porn that experts say can quickly spread widely online.

Digital privacy experts tried to raise some concerns, warning that the law seemed overly broad and could trigger widespread censorship online. Given such a short window to comply, platforms will likely remove some content that may not be NCII, the Electronic Frontier Foundation (EFF) warned. And even more troublingly, the law does not explicitly exempt encrypted messages, which could potentially encourage platforms to one day break encryption due to the liability threat. Also, it seemed likely that the removal process could be abused by people who hope platforms will automatically remove any reported content, especially after Trump admitted that he would use the law to censor his enemies.

None of that feedback mattered, the EFF’s assistant director of federal affairs, Maddie Daly, told Ars. Lawmakers accepted no amendments in their rush to get the bill to Trump’s desk. There was “immense pressure,” Daly said, “to quickly pass this bill without full consideration.” Because of the rush, Daly suggested that the Take It Down Act still has “gaping flaws.”

While the tech law is expected to achieve the rare feat of getting through Congress at what experts told Ars was a record pace, both supporters and critics also expect that the law will just as promptly be challenged in courts.

Supporters have suggested that any litigation exposing flaws could result in amendments. They’re simultaneously bracing for that backlash, while preparing for the win ahead of the vote tonight and hoping that the law can survive any subsequent legal attacks mostly intact.

Experts disagree on encryption threats

In a press conference hosted by the nonprofit Americans for Responsible Innovation, Slade Bond—who serves as chair of public policy for the law firm Cuneo Gilbert & LaDuca, LLP—advocated for the law passing, warning, “we should not let caution be the enemy of progress.”

Bond joined other supporters in suggesting that apparent threats to encryption or online speech are “far-fetched.”

On his side was Encode’s vice president of public policy, Adam Billen, who pushed back on the claim that companies might break encryption due to the law’s vague text.

Billen predicted that “most encrypted content” wouldn’t be threatened with takedowns—supposedly including private or direct messages—because he argued that the law explicitly covers content that is published (and, importantly, not just distributed) on services that provide a “forum for specifically user generated content.”

“In our mind, encryption simply just is not a question under this bill, and we have explicitly opposed other legislation that would explicitly break encryption,” Billen said.

That may be one way of reading the law, but Daly told Ars that the EFF’s lawyers had a different take.

“We just don’t agree with that reading,” she said. “As drafted, what will likely pass the floor tonight is absolutely a threat to encryption. There are exemptions for email services, but direct messages, cloud storage, these are not exempted.”

Instead, she suggested that lawmakers jammed the law through without weighing amendments that might have explicitly shielded encryption or prevented politicized censorship.

At the supporters’ press conference, Columbia Law School professor Tim Wu suggested that, for lawmakers facing a public vote, opposing the bill became “totally untenable” because “there’s such obvious harm” and “such a visceral problem with fake porn, particularly of minors.”

Supporter calls privacy fears “hypothetical”

Stefan Turkheimer, vice president of public policy for the anti-sexual abuse organization RAINN, agreed with Wu that the growing problem required immediate regulatory action. While various reports have indicated for the past year that the amount of AI-generated NCII is rising, Turkheimer suggested that all statistics are severely undercounting and outdated as he noted that RAINN’s hotline reports are “doubling” monthly for this kind of abuse.

Coming up for a final vote amid an uptick in abuse reports, the Take It Down Act seeks to address harms that most people find “patently offensive,” Turkheimer said, suggesting it was the kind of bill that “can only get killed in the dark.”

However, Turkheimer was the only supporter at the press conference who indicated that texting may be part of the problem that the law could potentially address, perhaps justifying critics’ concerns. He thinks deterring victims’ harm is more important than weighing critics’ fears of censorship or other privacy risks.

“This is a real harm that a lot of people are experiencing, that every single time that they get a text message or they go on the Internet, they may see themselves in a non-consensual image,” Turkheimer said. “That is the real problem, and we’re balancing” that against “sort of a hypothetical problem on the other end, which is that some people’s speech might be affected.”

Remedying text-based abuse could become a privacy problem, an EFF blog suggested, since communications providers “may be served with notices they simply cannot comply with, given the fact that these providers cannot view the contents of messages on their platforms. Platforms may respond by abandoning encryption entirely in order to be able to monitor content—turning private conversations into surveilled spaces.”

That’s why Daly told Ars that the EFF “is very concerned about the effects of Take It Down,” viewing it as a “massive privacy violation.”

“Congress should protect victims of NCII, but we don’t think that Take It Down is the way to do this or that it will actually protect victims,” Daly said.

Further, the potential for politicians to weaponize the takedown system to censor criticism should not be ignored, the EFF warned in another blog. “There are no penalties whatsoever to dissuade a requester from simply insisting that content is NCII,” the blog noted, urging Congress to instead “focus on enforcing and improving the many existing civil and criminal laws that address NCII, rather than opting for a broad takedown regime.”

“Non-consensual intimate imagery is a serious problem that deserves serious consideration, not a hastily drafted, overbroad bill that sweeps in legal, protected speech,” the EFF said.

That call largely fell on deaf ears. Once the law passes, the EFF will continue recommending encrypted services as a reliable means to protect user privacy, Daly said, but remains concerned about the unintended consequences of the law’s vague encryption language.

Although Bond said that precedent is on supporters’ side—arguing “the Supreme Court has been abundantly clear for decades that the First Amendment is not a shield for the type of content that the Take It Down Act is designed to address,” like sharing child sexual abuse materials or engaging in sextortion—Daly said that the EFF remains optimistic that courts will intervene to prevent critics’ worst fears.

“We expect to see challenges to this,” Daly said. “I don’t think this will pass muster.”

Photo of Ashley Belanger

Ashley is a senior policy reporter for Ars Technica, dedicated to tracking social impacts of emerging policies and new technologies. She is a Chicago-based journalist with 20 years of experience.

Trump’s hasty Take It Down Act has “gaping flaws” that threaten encryption Read More »

x-ignores-revenge-porn-takedown-requests-unless-dmca-is-used,-study-says

X ignores revenge porn takedown requests unless DMCA is used, study says

Why did the study target X?

The University of Michigan research team worried that their experiment posting AI-generated NCII on X may cross ethical lines.

They chose to conduct the study on X because they deduced it was “a platform where there would be no volunteer moderators and little impact on paid moderators, if any” viewed their AI-generated nude images.

X’s transparency report seems to suggest that most reported non-consensual nudity is actioned by human moderators, but researchers reported that their flagged content was never actioned without a DMCA takedown.

Since AI image generators are trained on real photos, researchers also took steps to ensure that AI-generated NCII in the study did not re-traumatize victims or depict real people who might stumble on the images on X.

“Each image was tested against a facial-recognition software platform and several reverse-image lookup services to verify it did not resemble any existing individual,” the study said. “Only images confirmed by all platforms to have no resemblance to individuals were selected for the study.”

These more “ethical” images were posted on X using popular hashtags like #porn, #hot, and #xxx, but their reach was limited to evade potential harm, researchers said.

“Our study may contribute to greater transparency in content moderation processes” related to NCII “and may prompt social media companies to invest additional efforts to combat deepfake” NCII, researchers said. “In the long run, we believe the benefits of this study far outweigh the risks.”

According to the researchers, X was given time to automatically detect and remove the content but failed to do so. It’s possible, the study suggested, that X’s decision to allow explicit content starting in June made it harder to detect NCII, as some experts had predicted.

To fix the problem, researchers suggested that both “greater platform accountability” and “legal mechanisms to ensure that accountability” are needed—as is much more research on other platforms’ mechanisms for removing NCII.

“A dedicated” NCII law “must clearly define victim-survivor rights and impose legal obligations on platforms to act swiftly in removing harmful content,” the study concluded.

X ignores revenge porn takedown requests unless DMCA is used, study says Read More »