This is pretty wild — our friends at "Read The Docs" saw file download traffic drop by 75% after blocking AI bots.
AI crawlers need to be more respectful
We talk a bit about the AI crawler abuse we are seeing at Read the Docs, and warn that this behavior is not sustainable.
Read the Docs (about.readthedocs.com)