Hey,
I'll post some stuff here regarding my migration from SMF 2.x forum to NodeBB. I've had a forum for ~4 years and it's about time I revamped it. I'm a huge node.js and redis fan, so having a forum written in php is blasphemous.
So far, i've been working on migrating the data from my SMF forum to NodeBB. Everything is going good here, i've been able to get everything into NodeBB with the correct timestamps etc etc. So that's cool.
NodeBB is crazy fast, and can be even faster.
One issue i've noticed so far, is that once I loaded some huge amounts of data into NodeBB, I started seeing a serious drop in performance. The good news is that we can fix it.
Here's a small screencast of what my 'load times' look like when visiting the "progress journals" subforum. This sub forum isn't "that big", 71 posts. However, some of the threads are enormous, so nodebb is doing say, 25k-30k redis operations just to load it, then it's doing a ton of server side html/dom processing which seems to be the biggest issue.
http://screencast.com/t/d1ooHZtEvf
Ok so, in order as is seen in the video, here are the "benchmarks".
- NodeBB unmodified code: 17.x seconds
- NodeBB modified code, using a redis.multi pipeline to send all of the hgetall's to redis in one request: 15.x seconds
- NodeBB modified code, using the same pipeline above AND getting rid of posts.toHTML(): 3.x seconds
**CORRECTION:
Also, all of that would actually be slower, but I've changed the redis code to use a UNIX DOMAIN SOCKET (/tmp/redis.sock) etc, which speeds it up a bit. I've also installed the hiredis native C redis package, that actually seems pretty significant as well.
**
I'm not so great with the DOM/"front end stuff", so i'm not sure how hard/easy/feasible it would be to move that posts.toHTML()/cheerio stuff to the browser, client side? If data were sanitized on the way in (during a post from a user), then you could simply return it to the browser and have it do the DOM crunching client side, which would free up nodebb server-side significantly?
So far, that seems to be the only issue i've come across.
Ok so i'll look for a few more spots to possibly speed it up initially. Right now i'm thinking the initial dashboard, it uses a bunch of hmget's maybe 'multi' will have an impact.
peace!
