Running NodeBB in clusters

mootzville

I am looking into running nodebb as a cluster. Does anyone have any experience with this, or recommended reading beyond the nodejs api docs?

Maybe one of the devs has a suggestion...?

If I have code like this:

var cluster = require( 'cluster' );
var path    = require( 'path' );
var root    = path.dirname( __dirname );
var cCPUs   = require('os').cpus().length;

if( cluster.isMaster ) {
  // Create a worker for each CPU
  for( var i = 0; i < cCPUs; i++ ) {
    cluster.fork();
  }

  cluster.on( 'online', function( worker ) {
    console.log( 'Worker ' + worker.process.pid + ' is online.' );
  });
  cluster.on( 'exit', function( worker, code, signal ) {
    console.log( 'worker ' + worker.process.pid + ' died.' );
  });
}
else {
   // Create master process used for forking here...
}

Any suggestions on what goes in the else statement?

Thanks,

Levi

<baris>

NodeBB already has support for clustering you can check out the loader.js file for more info.

mootzville

Also, I'd be interested in hearing anything about how others handle their load balancing...

mootzville

@baris Ok, great. I'm on 0.5.0-4, so looks like I need to step things up...

Thanks

julian

Yes, clustering was something we worked on over the past month, so you can scale up NodeBB to as many CPUs as you have on your machine!

mootzville

@julian Awesome. It's great to have it baked in like that. Also, after glancing at the loader.js file it looks like it automatically forks each cpu. I wanted to suggest allowing this to be a key/val that can be set in config.json...

Just a thought.

EDIT: It would need a conditional in case people -- optimistically -- set their cpuNum higher than what they actually had to default to cpuMaxNum and obviously a cpuMinNum of 1.

So, now if you are running 4 cores you can have two or three managing your nodebb process and one or two for your db...that is, until you're forum is so popular you need to upgrade to 64 cores and a mongod shard for each category

julian

@mootzville Actually, this is the case. You can add in a "cluster" property into your config.json, and it will spawn that many processes.

It doesn't check a max, so you could theoretically spawn 20 processes on a 1 CPU VPS... but that's your call, not mine

psychobunny

You can also override the config settings for testing purposes via:

node loader.js --cluster=X

Where X is the amount of cores you wish to utilise

mootzville

I'm impressed...thanks fellas