What Trust and Safety Teams Should Know About Online Grooming

Grooming is one of the hardest harms to catch at platform scale, and the reason is structural. Most abusive content moderation is built to judge a single artifact: an image, a video, a post. Grooming is not an artifact. It is a process that unfolds over many messages, often over days or weeks, and almost every individual message in that process looks harmless on its own. A compliment, a question about someone's day, an offer to talk somewhere more private. Read one at a time, none of these trip a rule. Read as a sequence aimed at a child, they describe an attack.

This article is for the people who build and run detection and moderation programs. It covers how grooming actually manifests as a signal problem, why point-in-time moderation misses it, what is worth modeling, and where automated detection and human review each belong.

Why message-by-message moderation misses it

Three properties make grooming resist conventional moderation.

It is benign at the message level. The abusive intent lives in the trajectory, not in any single line. A classifier or keyword rule that scores messages independently has almost nothing to grab onto early in a conversation, which is exactly when intervention matters most.

It spans time. The manipulation is spread across a conversation and often across sessions. That said, it does not always take long. WeProtect Global Alliance's 2023 Global Threat Assessment cites analysis finding the average time to engage a child in a high-risk grooming conversation was 45 minutes, with a minimum in gaming environments of 19 seconds. Detection has to reason over a window, not a single event, and sometimes that window is short.

It moves across platforms. A contact made in a game or a comment section gets pushed to a messaging app, then to a service with disappearing messages or a payment rail. The Tech Coalition, launching its cross-platform Lantern program in 2023, put the core problem plainly: because this activity spans platforms, any one company can often see only a fragment of the harm facing a victim. Offenders deliberately use several services in a single offense, and moving a conversation off-platform is itself a tactic.

Signals worth modeling

Academic models of grooming describe it as a set of stages rather than a single act. O'Connell's widely cited 2003 model breaks online grooming into friendship-forming, relationship-forming, risk assessment, exclusivity, and sexual stages. Later converged models describe victim selection, gaining access and isolation, trust development, desensitization, and post-abuse maintenance. You do not need to treat any one model as ground truth to draw a practical lesson from all of them: grooming has recognizable functional moves, and those moves are more detectable than any single word.

Signals that map to those moves and are worth modeling include:

• Rapport and trust building. NCMEC describes tactics such as developing rapport through compliments and discussing shared interests. On its own this is ordinary conversation, which is why it only becomes a signal in context.
• Isolation. Attempts to move a conversation off the platform, or to a private or ephemeral channel, are a strong behavioral signal.
• Risk assessment by the offender. Probing questions about who else can see the messages, whether a device is shared, or whether a parent is nearby.
• Desensitization. A gradual shift toward sexual conversation or role-play, which NCMEC lists as a grooming method.
• Extortion language. Payment requests or threats targeting a minor, which point to financial sextortion.

The pattern across all of these is that behavioral and relational signals carry more information than lexical ones. Codewords change. The move to isolate a child, to assess risk, or to extort does not.

The shift to self-generated imagery and sextortion

If your model of grooming ends at an offline meeting, it is out of date. A large and growing share of abuse material is what the Internet Watch Foundation calls self-generated, meaning content a child was groomed, deceived, or coerced into producing themselves, typically over a webcam or phone. The mechanism that produces this imagery is grooming, which makes text-based detection directly relevant to image-based harm.

Financial sextortion has become a distinct and fast-moving threat. IWF reporting for 2024 recorded a sharp rise in sextortion cases and noted that the targets are now heavily teenage boys. NCMEC reported receiving close to 100 reports of financial sextortion per day in 2024. WeProtect's briefing notes these schemes can move from a first message to coercion within hours, so the detection window can be very short. This is not a slow-burn grooming pattern, and programs tuned only for slow relationship building will be late.

The scale you are detecting against

The volumes explain why manual review alone cannot carry this. NCMEC's CyberTipline received more than 20 million reports in 2024. Online enticement reports, the category that includes grooming and sextortion, grew to more than 546,000 in 2024, a roughly 192% increase over 2023, and NCMEC has reported that growth continuing sharply into 2025. No human queue absorbs that. Automated detection is what makes the volume tractable, and human review is what makes the automated output safe to act on.

Where automation and human review each belong

The workable division of labor is narrow and worth stating precisely. Automated detection is for surfacing and prioritizing. It reads across a conversation window, scores behavioral and relational signals, and routes the highest-risk conversations to people. It should not take consequential action on its own.

Human review is for judgment and for anything that leaves the platform. Under US law, the platform is the reporting entity to the CyberTipline, and those reports are reviewed by people, at the platform and again at NCMEC, where analysts label content to help law enforcement prioritize. Reporting is not automatic and should never be described as such. A classifier can tell you where to look. A person decides what it means and whether it becomes a report.

This is also the honest answer to the accuracy problem. Grooming detection operates at very low prevalence, where even an accurate model produces false positives that a person has to filter. Building the human review step in from the start is not a compliance afterthought. It is what keeps the system both effective and fair.