Si vous ne souhaitez pas autoriser l'indexation, parfois intrusive, de votre site par des outils d'IA, il est possible de les bloquer sur votre hébergement :
Sur un hébergement WEBDEV - via le web.config :
<rewrite>
<rules>
<rule name="Blocage IA" stopProcessing="true">
<match url="^" ignoreCase="false" />
<conditions logicalGrouping="MatchAll" trackAllCaptures="false">
<add input="{HTTP_USER_AGENT}" pattern="(CCBot|ChatGPT|GPTBot|anthropic-ai|ClaudeBot|Omgilibot|Omgili|FacebookBot|Diffbot|Bytespider|PerplexityBot|ImagesiftBot|cohere-ai)" />
</conditions>
<action type="AbortRequest" />
</rule>
</rules>
</rewrite>
Sur un hébergement PHP - via le .htaccess :
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} (CCBot|ChatGPT|GPTBot|anthropic-ai|ClaudeBot|Omgilibot|Omgili|FacebookBot|Diffbot|Bytespider|PerplexityBot|ImagesiftBot|cohere-ai) [NC]
RewriteRule ^ – [F]