Storage costs
-
I am really liking these forums, and it would integrate nicely with several of my current nodejs projects. However, I have a few questions which may tilt me either into supporting it or whipping out my own super-simple one, this mostly comes down to storage situation.
Great choice with redis from a performance angle, but without knowledge of exactly how you store things in it, I am moderately concerned with overhead. Any rough estimates on storage overhead per user/post/etc (any specs would be helpful). I run a custom fork of redis to do geocoding, so downgrading to one that supports virtual memory is not-doable. This means I would need the forums, my nodejs site, and possibly mongodb with its big memory mapped files to all fit within 2GB of ram. I am using both ramnode and digitalocean SSD backed boxes for hosting. This information would help me make a decision as to viability.
Also, where/how are you storing images? Are these being serialized into redis, or are they being stored onto disk? Whereas I can no doubt store a shit-ton of text within a reasonable amount of ram, I could easily see thumbnails or user uploads getting out of hand. Any information there would be more then helpful.
Finally, do you have anything to shard on, or am I going to have to figure out something else in that regard?
Thanks
Richard Cesar -
also, what are you using for fallback? long polling? (socket.io etc)
-
Hello @Richard-Cesar, thanks for the comments!
Currently, for this forum, at 193 Topics and 905 Posts, our redis instance is using 6.78M, as reported by redis, with a memory fragmentation ratio of 1.19.
A clean install uses 433.88K.
The part of NodeBB that takes up the most space would be the actual post content, and to address this, we're building a set of "otherDB" plugins so that post content (at minimum) can be persisted to a disk-based data store, at which point the redis key containing the post data would have an
EXPIREset, allowing redis to function more like a supercache for posts.The same concept can be extended for topics and such, if the need arises.
User avatar uploads are stored to disk, while drag-and-drop image uploading is uploaded to imgur (if set up with an imgur key, otherwise the functionality is disabled).
For web socket fallbacks, we use the fallbacks as designed in socket.IO -- websockets fall back to xhr polling, then jsonp polling. I do not believe we enabled flashsockets, although I could be wrong.
