If you feel moderately safe touching your server config and want to block AI crawlers from your site, here's a few lines that will take care of many, even if they ignore robots.txt:https://github.com/ai-robots-txt/ai.robots.txt(Apache or nginx)(I perso...
-
If you feel moderately safe touching your server config and want to block AI crawlers from your site, here's a few lines that will take care of many, even if they ignore robots.txt:
https://github.com/ai-robots-txt/ai.robots.txt
(Apache or nginx)
(I personally set the return code to 444, no response, close the connection.)
Copyright © 2024 NodeBB | Contributors