Tech companies have an important role in limiting how their AI systems are used for exploitation. That responsibility can begin before a tool reaches the public and continue after its release through safety measures designed to detect prohibited activity. For some online service providers, federal law also requires reporting certain apparent child sexual exploitation offenses once they become aware of them.
The Law Office of Christopher St. John handles criminal defense matters involving technology and digital evidence in New Jersey, including AI child pornography allegations. Below, we look at the responsibilities tech companies have in preventing AI exploitation and responding when prohibited material is detected.
Building Safety Into AI Platforms
Preventing exploitation starts before an AI tool is released. Tech companies can screen training data for abusive material and test how models respond to attempts to generate prohibited content. Restrictions can also be built into the system before the tool reaches users. Testing can reveal whether users are able to bypass those restrictions or whether a model produces prohibited material in situations developers didn’t anticipate.
This approach, known as Safety by Design, treats protection as an ongoing responsibility rather than a one-time check before launch. New forms of misuse can happen after a system is released, so companies continue testing their models and updating safeguards as those risks change.
Detecting and Reporting Exploitative Content
Once a platform is in use, companies need ways to detect exploitation that bypasses their initial safeguards. Hash matching can recognize files that have already been identified as child sexual abuse material, while AI classifiers can flag potentially new material for further review. Some systems can also analyze conversations for signs of exploitation, including sextortion.
Federal law also sets reporting requirements for certain providers. Under 18 U.S.C. § 2258A, qualifying electronic service providers that obtain actual knowledge of apparent child sexual exploitation violations must report that information to the National Center for Missing & Exploited Children (NCMEC) through a CyberTipline report.
The report can include a name or email address associated with the account, along with details and content related to the suspected activity. Providers must also preserve the reported content.
Legal Guidance for AI-Related Allegations in New Jersey
Tech companies have their own responsibilities, but how someone uses an AI tool is evaluated separately. When AI-generated material becomes part of a criminal investigation, the person’s alleged conduct and the evidence connecting them to the material become central to the case.
Prosecutors can look at how the material was created or obtained and use available digital records to establish a connection between the content and the person under investigation. Have questions about an AI-related investigation or charge? Contact The Law Office of Christopher St. John or call 856-517-2958 for a free consultation.
Christopher St. John
Tess Berkowitz