Questions about the setup for community.nodebb.org

[baris]

6. Do you manually set the amount of memory for WiredTiger or just go with the default 50% - 1GB?

Default settings

7. Any special settings for redis?

in redis.conf

maxmemory 250000000 
maxmemory-policy volatile-lru

This limits redis to 250mb memory and evicts least recently used keys when memory is reaches 250mb.

8. Do you use nginx to serve static assets?

Yes

[PitaJ]

IIRC nginx also handles the 503 page

[baris]

Node instances use around ~220mb it can go higher depending on traffic.

[djensen47]

@baris Damn, mine is using 750MB+ but I guess half of that is closer to your number. I might have more plugins, which might effect the size in memory?

[djensen47]

Thanks for the info. It looks like I need 2 CPUs for the nodebb server and to setup a redis instance (somewhere).

Last thing, do 503s caused by load get logged? If not, how can I change it so that they are logged. I want to measure current 503 occurrences vs. any after I make these changes. Basically, I want to make sure throwing more money at the problem (more/better instances) fixes it.

[faizanzahid]

do you know the correct way to add clustering on node bb ?

[djensen47]

@faizanzahid Yes, it's in the docs: https://docs.nodebb.org/configuring/scaling/

Do not create more processes than CPUs. It's not advised as the results of doing such a thing are unpredictable.

[djensen47]

Ugh, I turned on clustering and now Socket.IO is broken ...

POST https://www.axisandallies.org/forums/socket.io/?EIO=3&transport=polling&t=MUiNOWE&sid=q1ikZjbjopxQbJZxAAYo 400 (Bad Request)

[djensen47]

I had to set the load balancing algorithm in HAProxy to source which is the same or similar to ip_hash in nginx.

[Tom_Rade]

@djensen47 What did you set it to?

[djensen47]

    balance source
    hash-type consistent

I've updated my haproxy config here: https://community.nodebb.org/topic/13315/haproxy-config

It probably needs a little more explanation that is there but it's an example to get you started.

[djensen47]

@baris How do I know if the pub/sub is working for sessions?

Some uses are reporting the logout problem after I turned on clustering and setup redis for pub/sub. But I haven't experienced it yet today but I haven't been on much.

Until I turned on IP sticky sessions, the site was erroring out on websockets (which might be unrelated).

[julian]

@djensen47 said in Questions about the setup for community.nodebb.org:

Until I turned on IP sticky sessions, the site was erroring out on websockets (which might be unrelated).

Not unrelated -- socket.io's handshake requires two requests that need to land on the same server, so the sticky sessions is a must

[djensen47]

@julian Thanks for the info! Given that it went away when I setup sticky sessions, that makes sense

It looks like redis is getting session keys. Does 120k session keys seem a little high?

[baris]

I think bots that hit the site generate a session key as well, so it might be right.

[djensen47]

@baris Any thoughts on how to prevent bots from generating a session?

[baris]

You can change saveUninitialized: true, to false to prevent bots creating sessions. https://github.com/expressjs/session/issues/325#issuecomment-229215641

[baris]

@djensen47 See https://github.com/NodeBB/NodeBB/issues/7115

I applied these changes on this forum as well, and spiders are no longer getting a session on every page hit.

[djensen47]

@baris Awesome!