Managing Crawlers - ChatGPT

[omega]

Is anyone looking at the ChatGPT and similar crawlers hitting their servers in terms of load?

If so have you put in place mitigations and do they work?

[baris]

I've opened a PR on the dependency we use so we can at least detect gptbot and deny access that way.

feat: add GPTBot by barisusakli · Pull Request #15 · binarykitchen/spider-detector

GitHub (github.com)

[omega]

@baris

[crazycells]

@baris are we able to see this gptbot in ACP?

[omega]

I propose we classify this traffic as a synthetic-entity traffic, if anyone has a better handle on the correct classification term, speak now.