503 - temporarily unavailable due to excessive load

[teh_g]

I know there was some discussion a few months back in this thread: https://community.nodebb.org/topic/6957/toobusy-excessive-load, but I wanted to see what actually triggers it. I understand that it is due the event loop lag going past a certain threshold. If I throw more hardware at the forum will that resolve this?

I am running my server on the 512MB droplet on DigitalOcean. Not sure if I have "too much" traffic for that level. nodejs is using about 45% of the memory, node is at 13% and redis-server is at 6.6%.

The forum is new, so the stats might be a bit blown up, but I am seeing about 11,000 page views a day, 400+ posts a day, 130 unique visitors, and 20+ topics a day.

So this leads me to a few questions:

• Is there a decent rule of thumb for when I need to move to a beefier box?
• Is there something for when clustering may be a better idea?

[rod]

Have you made adjustments to the defaults at ACP > Settings > Advanced > Traffic Management?

[teh_g]

@rod said in 503 - temporarily unavailable due to excessive load:

Have you made adjustments to the defaults at ACP > Settings > Advanced > Traffic Management?

I have not made changes there. I am all for increasing those values to prevent the messages, but I want to make sure a better route isn't to just scale up the hardware.

[pichalite]

I think you should at least get more memory. based on your stats... at 400+ posts a day you will need more memory to keep using redis as it's more of a in-memory database.

[teh_g]

@pichalite said in 503 - temporarily unavailable due to excessive load:

I think you should at least get more memory. based on your stats... at 400+ posts a day you will need more memory to keep using redis as it's more of a in-memory database.

Is 1 GB enough? I can't find a rule of thumb kind of guide for x posts -> y memory.

Is it worth switching to MongoDB over redis? I've been poking around and apparently Mongo is what people seem to prefer here.

[pichalite]

@teh_g I don't have any stats to show how much memory is required for x number of posts. I would say increase the memory to 1 GB. Keep monitoring memory usage and make changes accordingly.

Is it worth switching to MongoDB over redis?

I am not an expert on that but people do seem to prefer mongo here.

[teh_g]

@pichalite said in 503 - temporarily unavailable due to excessive load:

I am not an expert on that but people do seem to prefer mongo here.

Hmm, I may look into switching in the future. I don't expect too much more traffic. Worst comes to worst, I will have to do a transfer from redis to mongo

[scottalanmiller]

@teh_g said in 503 - temporarily unavailable due to excessive load:

@rod said in 503 - temporarily unavailable due to excessive load:

Have you made adjustments to the defaults at ACP > Settings > Advanced > Traffic Management?

I have not made changes there. I am all for increasing those values to prevent the messages, but I want to make sure a better route isn't to just scale up the hardware.

We use 6 cores, 8GB and all SSD with Linode (which is a bit faster per core than DO) and we still get the 503 as well. Posts are not a great indicator of load. We get the 503 based on heavy read traffic, not post rate.

[scottalanmiller]

@teh_g said in 503 - temporarily unavailable due to excessive load:

@pichalite said in 503 - temporarily unavailable due to excessive load:

I think you should at least get more memory. based on your stats... at 400+ posts a day you will need more memory to keep using redis as it's more of a in-memory database.

Is 1 GB enough? I can't find a rule of thumb kind of guide for x posts -> y memory.

Is it worth switching to MongoDB over redis? I've been poking around and apparently Mongo is what people seem to prefer here.

1GB is not enough. I would get to 1.5GB minimum. And yes, Mongo is great. We use Mongo for scalability. It's been great.

[teh_g]

@scottalanmiller said in 503 - temporarily unavailable due to excessive load:

1GB is not enough. I would get to 1.5GB minimum. And yes, Mongo is great. We use Mongo for scalability. It's been great.

Interesting. I would love if we made a "community guideline" for a rough estimation of memory to activity ratio.

@scottalanmiller said in 503 - temporarily unavailable due to excessive load:

We use 6 cores, 8GB and all SSD with Linode (which is a bit faster per core than DO) and we still get the 503 as well. Posts are not a great indicator of load. We get the 503 based on heavy read traffic, not post rate.

What is your load like? I am at about 11k page views a day. For Mongo, I will have to look into getting nodeBB to make the migration for me, since it seems they own the migration script.

[scottalanmiller]

@teh_g said in 503 - temporarily unavailable due to excessive load:

@scottalanmiller said in 503 - temporarily unavailable due to excessive load:

1GB is not enough. I would get to 1.5GB minimum. And yes, Mongo is great. We use Mongo for scalability. It's been great.

Interesting. I would love if we made a "community guideline" for a rough estimation of memory to activity ratio.

Well here is what I know..... 2 million page loads / 180 million hits per month is using about 5.5GB with cache and buffers included.

[scottalanmiller]

@teh_g said in 503 - temporarily unavailable due to excessive load:

What is your load like? I am at about 11k page views a day. For Mongo, I will have to look into getting nodeBB to make the migration for me, since it seems they own the migration script.

NodeBB reports two million views per month.

CloudFlare in front of NodeBB reports 180 million requests per month with a huge cache hit success rate. So only about half make it to the server at all. The CF number climbed from 10 million to 180 million with little to no impact on the NodeBB reporting numbers (they actually decreased, not increased.)

We run 500 - 600 posts per day.

[scottalanmiller]

So our daily views (as reported by NodeBB) is 32K - 98K.

[teh_g]

@scottalanmiller said in 503 - temporarily unavailable due to excessive load:

So our daily views (as reported by NodeBB) is 32K - 98K.

It sounds like I should be a little amazed that 512 MB of memory works...

I'll have to scale up my Digital Ocean tonight...