Scaling NodeBB on Google Cloud - Is this approach correct?

shri

Loving everything I see with NodeBB and before I commit to it, I want to test its scalability on a DIY cloud setup on Google.

This is the setup that I'm looking at:

NFS / REDIS / MongoDB machine with two storage disks. Redis and MongoDB disk will be a low capacity (30-60GB) SSD. The NFS share will be a 1TB disk (doubt I'll need it .. but disks are cheap and don't want to outgrow it in our use case).
NodeBB - this will be a NGINX + NodeJS instance which will run the NodeBB from an NFS mount. I figure with every 2GB RAM instance we should be able to fire up about 10 NodeJS instances on that machine and NGINX will "proxy" to those instances on a iphash. This "Nginx+NodeJS" instance, once working with NodeBB, will be added to an autoscale group which will fire up more instances as and when required. (We typically add new instances when average load hits 70%).
A Google https load balancer will receive the traffic and direct to an appropriate instance.

NodeBB will be configured to use MongoDB as the database and will have a redis config (I read somewhere that redis is required for the push messaging to work.

Am I on the right track with scaling this?

I assume given the little time that I've spent on NodeBB all the NodeJS modules, plugin themes and other dependencies are installed in the NodeBB private directory and not global to NodeJS (i.e. in my specific case stored on the NFS share).

One small area that I've not investigated is restarting / reloading NodeBB on multiple instances. How would this work? Would we have to script this to ssh into each instance and do a ./nodebb restart or would it work from the admincp?

(I do want to stick with Google Cloud - so suggestions of other services will not work ... I have to make this work on GCE!)

julian

@shri said:

I read somewhere that redis is required for the push messaging to work.

Not required, but highly recommended when working with MongoDB, since Mongo's more efficient at reading, less so at writing, due to document-level locking.

shri

Thanks for the clarification.

So based on the lack of negative responses or correction, I'll assume that this is the right path to take.

<baris>

Actually if you have more than 1 nodebb instance and you don't use redis then socket.io events won't reach clients that are connected to different nodebb instances. So in that case you should use redis you will get a warning if you don't.

Let us know how it goes.

JonnyBlack

Interesting build out. regular updates! :metal:

shri

It is taking me some time - mostly because I'm not familiar enough with NodeJS / NodeBB.

Some observations:

Took a while to get the ./nodebb setup routine to work - had to do it first for MongoDB and then for Redis and then combine the two files and restart (redis / mongodb nodejs files do not get installed on the npm install command).

Then got stuck with launching multiple processes on reboot with systemd. Gave up on that and just used a crontab @ reboot line to run a shell script. Need to figure out how to restart / reload processes seamlessly.

Found some issues with socket.io and spdy - did not investigate, disabled spdy on nginx.

Found some issues with the 0.7.0 branch and upgraded to master 0.7.1 and found that much more stable. But these were plugin related - sections, adsense - dropped them both.

Made the mistake of setting the production flag in the environment variable NODE_ENV.

As of now, have a test site running persona, notifications plugins on a distributed setup - NFS / REDIS / MONGO DB on one server and a barebones NODEJS + NGINX/SSL on another.

Seems stable for now on a test setup on https://devlib.org will give it a day or two while I play around with the templates / plugins and watch memory usage, before I go through another effort to set the autoscale / loadbalancer up.

@baris @julian @psychobunny - I really like what I see so far. Well done. If all goes well with the setup / eval, I will be dropping you guys an email to see if I can fund some development.