Community

Storage requirements

Solved General Discussion
  • A

    Hi there,

    I am planning on using NodeBB and was wondering how much space would around 10 million posts and 1 million users work out at? As using a VPS can cost a bit (storage-wise) and self-hosted can be relatively cheap, depending how much space the above mentioned will take.

    If anybody can help me out with answering that, that would be great. I am in between NodeBB and Flarum, hence the question. I haven't gotten an answer at Flarum about this, though.

    Scalability is what I want to know. Resources and performance I can figure out myself, but storage is hard to determine.

    Thanks and kind regards,

    exodoE 1 Reply
    1
  • A

    Or if someone can give me the average size of 1 post (in bytes, or KB, or MB, etc) and the average size of 1 user (in bytes, or KB, or MB, etc), that would be helpful.

    0
  • A

    bump

    Anybody?

    0
  • barisB

    Your main post and user object in bytes

    >Object.bsonsize(db.objects.find({_key:"post:54280"}));
795
> Object.bsonsize(db.objects.find({_key:"user:9534"}));
513

    You can also go to /admin/advanced/database to see # of documents, average document size and storage size there.

    A 1 Reply
    0
  • A

    @baris Thank you so much, that was what I was looking for. It uses less space than Flarum does. I think I will be sticking with NodeBB then.

    Also, thanks for pointing out that directory, that will come in handy 🙂

    BriB 1 Reply
    0
  • BriB

    @Adam-Poniatowski said in Storage requirements:

    Also, thanks for pointing out that directory, that will come in handy 🙂

    That was actually a url, just FYI 🙂

    A 1 Reply
    1
  • A

    @Bri URL, directory... its all the same to me 🙂

    0
  • exodoE

    @Adam-Poniatowski said in Storage requirements:

    Hi there,

    I am planning on using NodeBB and was wondering how much space would around 10 million posts and 1 million users work out at? As using a VPS can cost a bit (storage-wise) and self-hosted can be relatively cheap, depending how much space the above mentioned will take.

    If anybody can help me out with answering that, that would be great. I am in between NodeBB and Flarum, hence the question. I haven't gotten an answer at Flarum about this, though.

    Scalability is what I want to know. Resources and performance I can figure out myself, but storage is hard to determine.

    Thanks and kind regards,

    700k posts and 24k topics

    MongoDB Version 3.0.12
Uptime in Seconds 1,857,623
Storage Engine mmapv1
Collections 7
Objects 7,613,327
Avg. Object Size 314.33 b
Data Size 2282.22 mb
Storage Size 4013.56 mb
Index Size 2261.97 mb
File Size 8125.00 mb
Resident Memory 466 mb
Virtual Memory 17860 mb
Mapped Memory 8541 mb
 REDIS
Redis Version 3.0.7
Uptime in Seconds 1,856,844
Uptime in Days 21
Connected Clients 31
Connected Slaves 0
Blocked Clients 0
Used Memory 241.97M
Memory Fragmentation Ratio 0.99
Total Connections Received 2,997
Total Commands Processed 6,200,287
Instantaneous Ops. Per Second 1
Keyspace Hits 2,728,703
Keyspace Misses 14,130
    2

Suggested Topics

  • macfanM

    Is CLA still required to contrbute to NodeBB?

    General Discussion
    0 Votes
    6 Posts
    391 Views

    julianJ

    @macfan that's understandable, I know not everybody is comfortable signing their contributions away, so they are welcome to fork NodeBB for their own purposes.

    Contributions back upstream are welcome and encouraged but not required 🙂

  • ButonixB

    Requires a theme developer!

    Moved General Discussion
    0 Votes
    2 Posts
    262 Views

    youhosiY

    Hello @Butonix, write in pw.

  • F

    GDPR compliance and storage of IP addresses

    General Discussion
    0 Votes
    13 Posts
    4062 Views

    julianJ

    Thanks @Juan-G -- I wanted to say as such to @frgilb but could not find the appropriate clauses 😄

    Here are some additional clauses (thanks @Jay-Moonah for looking into this earlier this week):

    “Processing shall be lawful only if and to the extent that at least one of the following applies: […] (f) processing is necessary for the purposes of the legitimate interests pursued by the controller or by a third party, except where such interests are overridden by the interests or fundamental rights and freedoms of the data subject which require protection of personal data, in particular where the data subject is a child.”

    -- Article 6, Paragraph 1, Point F

    “The processing of personal data to the extent strictly necessary and proportionate for the purposes of ensuring network and information security, i.e. the ability of a network or an information system to resist, at a given level of confidence, accidental events or unlawful or malicious actions that compromise the availability, authenticity, integrity and confidentiality of stored or transmitted personal data, and the security of the related services offered by, or accessible via, those networks and systems, […] by providers of electronic communications networks and services and by providers of security technologies and services, constitutes a legitimate interest of the data controller concerned. This could, for example, include preventing unauthorised access to electronic communications networks and malicious code distribution and stopping ‘denial of service’ attacks and damage to computer and electronic communication systems.”

    -- Recital 49 (excerpt)

    That said, where IP address is used in a fashion that isn't exposed to anyone of significance (regular users or admins), then I see no reason to utilise the IP, keep it for any lengthy period of time, or at least secure it properly.

    To that end, please see gh#6539 (attached) to see how I've addressed the first point.

    I would argue that the storage of IP addresses per user (via User.logIP()) is required in order to prevent unauthorized access or cyber-attacks, although I use that term fairly loosely. I've identified the following use cases:

    Admin approval for registration (if an IP is already associated with a uid) -- useful for combating sockpuppetry Get similar uids during admin approval stage -- again, sockpuppetry-mitigation Search by IP -- used by moderators to find existing sockpuppets.

    With GDPR consent required for all users, this is no longer an issue as they would be consenting to their storage of IP addresses for this purpose, and we do delete on user deletion, so this satisfies the "Right to be Forgotten".

  • K

    redis does not require a password, but password was supplied

    General Discussion
    0 Votes
    6 Posts
    5375 Views

    teh_gT

    @kenygamer said in redis does not require a password, but password was supplied:

    @teh_g No.

    I'd ignore it then. I think it might be tied to this issue that was fixed in a more recent node-redis version: https://github.com/dwyl/redis-connection/issues/31

  • babaggeiiB

    Memory requirements for ~100 concurrent users

    General Discussion
    0 Votes
    12 Posts
    5764 Views

    NivixN

    @julian I've got to say--the documentation for NodeBB is why I chose it over Discourse. There are still some docs lacking (and I know since the platform has not reached a full "1.0" version we should not expect full docs)...but the fact that you guys have put work into writing some documentation is great.

    Thanks so much!

| | | |
Copyright © 2023 NodeBB | Contributors