TikTok aware of risks to teens, court document alleges

TikTok was aware that its design features are detrimental to its young users and that tools aimed at limiting kids’ time on the site were largely ineffective, according to internal documents in a lawsuit filed by the US state of Kentucky.

The details are among redacted portions of Kentucky’s lawsuit that contains the internal communications and documents unearthed during a more than two year investigation into the company by various US states.

Kentucky’s lawsuit was filed this week, alongside separate complaints brought forth by attorneys general in a dozen states as well as the District of Columbia.

The redacted information – which was inadvertently revealed by Kentucky’s attorney general’s office – touches on a range of topics, most importantly the extent to which TikTok knew how much time young users were spending on the platform and how sincere it was when rolling out tools aimed at curbing excessive use.

The unredacted complaint, which was seen by The Associated Press, was sealed by a Kentucky state judge on Wednesday after state officials filed an emergency motion to seal it.

When reached for comment, TikTok spokesperson Alex Haurek said: “It is highly irresponsible of the Associated Press to publish information that is under a court seal. Unfortunately, this complaint cherry-picks misleading quotes and takes outdated documents out of context to misrepresent our commitment to community safety.”

“We have robust safeguards, which include proactively removing suspected underage users, and we have voluntarily launched safety features such as default screen time limits, family pairing, and privacy by default for minors under 16,” Haurek said in a prepared statement. “We stand by these efforts.”

The complaint alleges TikTok has quantified how long it takes for young users to get hooked on the platform, and shared the findings internally in presentations aimed at increasing user-retention rates.

The “habit moment,” as TikTok calls it, occurs when users have watched 260 videos or more during the first week of having a TikTok account. This can happen in under 35 minutes since some TikTok videos run as short as eight seconds, the complaint says.

TikTok does its own internal studies to find out how the platform is impacting users. The lawsuit cites one group within the company, called “TikTank,” which noted in an internal report that compulsive usage was “rampant” on the platform.

TikTok has a 60-minute daily screen time limit for minors, a feature it rolled out in March 2023 with the stated aim of helping teens manage their time on the platform. But Kentucky’s complaint argues the time limit – which users can easily bypass or disable – was intended more as a public relations tool than anything else.

The lawsuit says TikTok measured the success of the time limit feature not by whether it reduced the time teens spent on the platform, but by three other metrics – the first of which was “improving public trust in the TikTok platform via media coverage.”

Reducing screen time among teens was not included as a success metric, the lawsuit said. In fact, it alleged the company had planned to “revisit the design” of the feature if the time-limit feature had caused teens to reduce their TikTok usage by more than 10 per cent.

In a section that details the negative impacts TikTok’s facial filters can have on users, Kentucky alleges that TikTok’s algorithm has “prioritised beautiful people” despite knowing internally that content on the platform could “perpetuate a narrow beauty norm.”

The lawsuit also takes aim at TikTok’s content-moderation practices.

It cites internal communication where the company notes its moderation metrics are “largely misleading” because “we are good at moderating the content we capture, but these metrics do not account for the content that we miss.”

The complaint notes that TikTok knows it has – but does not disclose – significant “leakage” rates, or content that violates the site’s community guidelines but is not removed or moderated.

For TikTok, the complaint notes the “leakage” rates include roughly 36 per cent of content that normalises pedophilia and 50 per cent of content that glorifies minor sexual assault.