OpenAI launches a system that detects AI abuse without storing customer data

OpenAI presented a new monitoring system that aims to detect improper uses of its models without retaining customer data. The tool, called Private Safety Processing, is particularly aimed at companies that handle sensitive information.

The new mechanism expands the scope of Zero Data Retention and allows for the analysis of interactions between different conversations. The proposal comes as OpenAI and Anthropic compete for the corporate market of artificial intelligence.

How OpenAI's new privacy system works

OpenAI introduced Private Safety Processing as an automated system that monitors potential misuse of its models. Its main feature is that it performs the analysis without retaining customer data.

The technology expands the scope of Zero Data Retention (ZDR), a policy that already allows monitoring for potential abuses without retaining company data.

While ZDR analyzes activities by session, the new system can evaluate the inputs and outputs of multiple conversations. In this way, it seeks to detect patterns of misuse that could remain hidden if each interaction is analyzed separately.

What happens if OpenAI detects potential misuse

The system operates through an agent that analyzes interactions and can be activated upon certain signs of abuse. According to OpenAI, it does not require human review of user conversations.

When activated, the tool can send the company a specific and limited signal about the type of activity detected. From that notice, OpenAI can determine whether any action should be taken.

If more information is needed, the company can reach out to the customer to understand the context. The customer can decide whether to share additional data with OpenAI.

The difference between OpenAI and Anthropic in data handling

The proposal from OpenAI contrasts with a recent policy from Anthropic for certain models. The company allows user data from the so-called covered models to be retained for 30 days.

This group includes models from the Mythos class and future models with similar capabilities. The policy was presented with security objectives and allows for the analysis of potential misuse.

Anthropic also establishes that there may be human review of customer data through controlled access. The company notes that a small group of authorized reviewers participates and that each session is recorded in a log protected against modifications.

What Private Safety Processing aims to detect

According to a spokesperson from OpenAI speaking to TechCrunch, the new technology aims to identify malicious uses distributed across multiple sessions.

A report cited in the material noted that OpenAI's growth during the second quarter was slower than that of Anthropic. Additionally, it was reported that Anthropic reached an annualized revenue run rate of US$65 billion.

Anthropic's investors also indicated that the company could reach a valuation of US$2 trillion in a potential IPO. At the same time, OpenAI is working on its own process for a future IPO.