My forum is using MongoDB and I notice nodebb keeps bunch of data on socket.io collection. Can I clean them?
crawlers asking for socket.io forever
I just remembered, when I finished doing my tests with NodeBB, some crawlers kept checking on socket.io (or something like that) forever, more than a month after I took it down.
Not enough to cause any load worth of being concerned about, just wanted to report it.
more than a month after I took it down.
are they still asking?
it should be about 6 months it's been down now if i recall correctly.
Odd... most crawlers don't even parse js, so it's probably GoogleBot, no?
In that case, perhaps you can make them go away with a 302 to send them someplace else that could use the SEO juice...