Mongodb support

<baris>

Hey everyone,

Just letting everyone know that we merged a big branch into NodeBB on friday. This new codebase allows you to use MongoDB as the database for NodeBB.

Some people had concerns about Redis as it is an in-memory database which means you need to have enough memory to fit all your data. Now you can pick which database to use during installation(node app --setup). The installation will ask you which database to use, the options are "redis"(default) and "mongo".

For people already running on Redis this doesn't change anything but if you are just installing NodeBB you have the option to pick MongoDB.

Further down the road we might write export/import tools between Redis and MongoDB so you can switch between the two without losing data.

Now some technical details about the change for developers :

We used to have code in NodeBB that used the redis object directly as follows :

var RDB = require('./redis');

RDB.hmset('user:' + uid, userData);

Now the same functionality looks like this after the merge :

var db = require('./database');

db.setObject('user:' + uid, userData);

NodeBB's code no longer deals directly with Redis or MongoDB. We just make a call to the database object to save an object and depending on which database you picked during installation the object will be saved in Redis or MongoDB.

If you check out the contents of database.js it is very simple :

var nconf = require('nconf'),
    databaseType = nconf.get('database'),
    winston = require('winston');

if(!databaseType) {
    winston.info('Database type not set! Run node app --setup');
    process.exit();
}

var db = require('./database/' + databaseType);

module.exports = db;

It reads the database type from nconf and then requires the correct database file from the database folder which can be either datatase/redis.js or database/mongo.js.

Both redis.js and mongo.js expose the same set of functions for example there is a setObject function in both redis.js and mongo.js :

// from redis.js
module.setObject = function(key, data, callback) {
    redisClient.hmset(key, data, function(err, res) {
        if(callback) {
            callback(err, res);
        }
    });
}



// from mongo.js
module.setObject = function(key, data, callback) {
    data['_key'] = key;
    db.collection('objects').update({_key:key}, {$set:data}, {upsert:true, w: 1}, function(err, result) {
        if(callback) {
            callback(err, result);
        }
    });
}

All NodeBB sees is the setObject(key, data, callback) method which takes a key and data and saves it in the respective database.

Since we started out with Redis we make use of the special Redis data structures namely hash, set, sorted set and list. All the functions exposed from the database files implement these data structures.

You can check out redis.js here and mongo.js here.

So hopefully this will be useful for people who want to use another database than Redis. Another cool thing is if we decide to add another database let's say CouchDB, all we have to do is add couch.js into the database folder and implement the same interface. Add it as an option in the install steps and there is no need to change any code in NodeBB itself.

Let us know what you think.

bentael

Awesome ! even though that broke almost any nodebb-* thing I wrote, but for the greater good ! and not for too long! Thanks for abstracting every "redis"-like function in the mongo module.

nanoVapor

That's awesome! I'll use this new version on my project later

julian

@bentael In cases where you are requiring the RDB object directly, yes, it would break (as it did in the case of the markdown plugin).

Best bet is to rely on built-in methods, if at all possible:

// To simulate a reply
var Posts = module.parent.require('./posts');
Posts.reply(tid, uid, 'derp derp', function() { ...

julian

Case in point -- in my markdown plugin, I am relying on querying the config hash, when I really should've been using module.parent.require('./meta') instead!

bentael

gotcha! [comment to short error? not anymore]

<baris>

Was getting an error with the nodejs driver with the latest version (1.3.23). Downgrading to 1.3.19 seems to fix the issue.

https://github.com/mongodb/node-mongodb-native/releases

https://groups.google.com/forum/#!topic/node-mongodb-native/EpyYqhTl-Ak

julian

Both of those versions are the same minor version (1.3.x) -- Mongo, I am disappoint.