@normando said in How to reduce the load?:
Anyway, seems crawler bots.
Do you have the logs? Is it primarily googlebot? It will come in like a thundering herd of thunderin' hoovers. Does not respect conventions, e.g. robots.txt, but rather requires you to sign up for their webmaster tools. Anyways, back to the relevance, if not, then googlebot will basically scale to suck up as much as it can as fast as it possibly can. Hence, if you are running NodeBB on a VM in some reasonably well connected data center, network bandwidth is unlikely to be the bottleneck, leaving the vm itself.
The googlebot comes in fast, but also leaves fast. Several times a day, at +/- regular intervals depending on site activity and its mystery algos. So if you are seeing more off the wall bots, at random times 'twixt visits, then maybe these bots warrant further investigation. Otherwise, not much you can do about goog bot. But maybe give it more resources and it can leave sooner.