OpenAI backs a shared AI-training signal—but website owners should not treat it as live yet

In its October 30 announcement, OpenAI says it intends to honor a shared robots.txt signal so operators can communicate the same training preference to participating providers. Today, this remains a commitment to implement in the coming months, with details promised later. The company also says its existing crawler controls will continue to apply. A future implementation promise is not evidence of working enforcement now.

The linked individual Internet-Draft proposes the token __usage_ai-training. In its example, User-agent: __usage_ai-training combined with Disallow: / expresses a preference against model training. But the IETF Datatracker expressly says this individual document lacks formal IETF endorsement and standing in the standards process. Its details may change. A robots.txt preference also cannot physically prevent copying, and no universal provider compliance is demonstrated here.

The most consequential distinction is between model training, which changes model parameters; crawling, which fetches pages; and AI search or agent retrieval, which may retrieve a page in response to a user task. The draft says its training policy token is not a crawling-control directive. That means operators can reasonably want different outcomes for model training and discoverability. Allowing training does not guarantee a citation, and expressing an objection does not automatically block live AI-assisted search.

The following table ties documented statuses to practical operator decisions, separating the vendor announcement, the draft's technical scope and our own editorial judgment. Operators should avoid accidentally suppressing search crawling when their actual objective relates only to AI model training. We have not independently tested enforcement, measured ranking changes or obtained legal confirmation about the effect of these declarations.

1. Audit: Inventory current robots.txt controls and compare them with OpenAI's existing crawler documentation.
2. Decide: Set training preferences for different content categories independently of search access, consulting relevant technical and legal owners.
3. Verify later: Track the individual draft and implementation updates. Version configuration changes and confirm actual requests in server logs instead of assuming a public promise is enforced.

A publisher may object to training while wanting its articles to remain discoverable to AI-assisted search. That is a different choice from blocking all automated access. AgentenCode consequently separates source access, editorial evidence and verified product claims. The proposed common signal could simplify the web's governance landscape, but it is not yet a universal legal right, an operational security barrier or a visibility guarantee.