Content Signals declared in robots.txt

medium

agent_bot_content_signals

Checks if robots.txt declares AI content usage preferences via Content Signals.

What does this rule check?

Content Signals are directives in robots.txt that declare your preferences for how AI systems may use your content — whether for training, search indexing, or as AI input. This is a proposed standard for granular AI content governance.

Why is this important?

Without Content Signals, AI systems have no machine-readable way to know your content usage preferences. Explicit signals prevent unauthorized use of your content for AI training while allowing beneficial uses like search.

How to fix

Add Content-Signal directives to your robots.txt declaring preferences for ai-train, search, and ai-input.

Examples

Incorrect 
User-agent: *
Allow: /

# No Content-Signal directives
Correct 
User-agent: *
Allow: /

Content-Signal: ai-train=no, search=yes, ai-input=no

Agent fix snippet

Copy this snippet and paste it into your AI coding agent to fix this issue automatically.

Add Content-Signal directives to your robots.txt:

Content-Signal: ai-train=no, search=yes, ai-input=no

See contentsignals.org for the full specification.