Mongodb or PostgreSQL? Ah, that is the question...
-
@gotwf said in Mongodb or PostgreSQL? Ah, that is the question...:
@scottalanmiller said in Mongodb or PostgreSQL? Ah, that is the question...:
@gotwf said in Mongodb or PostgreSQL? Ah, that is the question...:
The Guardian is using AWS's proprietary enhanced PostgreSQL
Yeah, my guess is that a lot of their decision there was based around being on AWS and wanting a hosted SaaS based database rather than running their own.
Hopefully the daily wtf notice this thread and weighs in w/their $0.02. Pretty please?
I assume that they are around on here, but don't know for sure.
-
I guess I'd like to go with Mongo, but posts like this, especially since appear to be unresolved, or if resolved, not updated to clue in the rest of us.
https://community.nodebb.org/topic/12510/mongodb-crashes-sometimes
https://community.nodebb.org/topic/13574/nodebb-v1-11-1-suddenly-started-crashing/9
Combined w/the stuff I linked earlier... suffice to say like this makes an old grey beard very nervous.
-
@gotwf said in Mongodb or PostgreSQL? Ah, that is the question...:
I guess I'd like to go with Mongo, but posts like this, especially since appear to be unresolved, or if resolved, not updated to clue in the rest of us.
https://community.nodebb.org/topic/12510/mongodb-crashes-sometimes
https://community.nodebb.org/topic/13574/nodebb-v1-11-1-suddenly-started-crashing/9
Combined w/the stuff I linked earlier... suffice to say like this makes an old grey beard very nervous.
Well, from a quick look...
The first is purely an unknown. There was just zero info in that thread, heaven only knows if MongoDB had a problem at all. He just mentioned that it wasn't running and didn't follow up. So that should be thrown out, it's not indicative of anything.
The second is a real issue, but really just tells us that like any database, it needs to be maintained. NodeBB doesn't require the latest and greated, but it does require relatively currently. Which you would want anyway. There is cause for concern, of course, as data was lost. But as a completely isolated case during an update, it seems like something to note, but not really worry about. The issue could easily not have been on MongoDB's side, especially given the resolution.
-
@gotwf said in Mongodb or PostgreSQL? Ah, that is the question...:
Hopefully the daily wtf notice this thread and weighs in w/their $0.02. Pretty please?
@Ben-Lubar from TDWTF built the pgSQL adaptor that NodeBB uses, actually. They're on postgres at the moment.
-
@julian said in Mongodb or PostgreSQL? Ah, that is the question...:
@gotwf said in Mongodb or PostgreSQL? Ah, that is the question...:
Hopefully the daily wtf notice this thread and weighs in w/their $0.02. Pretty please?
@Ben-Lubar from TDWTF built the pgSQL adaptor that NodeBB uses, actually. They're on postgres at the moment.
And it is noticeably a lot slower. But that could be any number of factors, no reason to assume that it is the database. But for it to be the example "less responsive" previously, and then it turns out to be the PostgreSQL example, there is a reasonable chance that the DB is the thing making it slower.
-
Right. Performance could be relative to resource utilisation, however. I know TDWTF had huge troubles with web scraper bots hammering the server to crawl their new content. They even blacklisted a number of user agents for that reason.
Might be the db, or might just be lots of bots crawling the site at any given time
-
@julian said in Mongodb or PostgreSQL? Ah, that is the question...:
Right. Performance could be relative to resource utilisation, however. I know TDWTF had huge troubles with web scraper bots hammering the server to crawl their new content. They even blacklisted a number of user agents for that reason.
Might be the db, or might just be lots of bots crawling the site at any given time
Very possible. Although we've seen some pretty heavy hammering from web crawlers in the past, but the site was able to take it. So even if it is web crawls, that might just expose that the DB can't keep up as well. It's a super popular site, and there are lots of factors. How are they caching, how much traffic are they seeing, how many writes are going on, what does the CPU, RAM, IO look like, what NodeJS version on they on, etc.
-
Well ain't I a lamer... Does not seem that long since I toyed with MongoDB on SmartOS but Mongo has dropped support since 2017.
https://www.theregister.co.uk/2017/08/29/mongodb_quits_solaris/
Fsck'ing Penguinistas. Too bad, too sad that their junior devs weren't smart enough to pick up a bit of real Unix.
Sure, I could spin up a Linux vm, but I already been there, done that and there are good reasons I prefer SmartOS (yeah, even over fbsd server side).
Thankfully NodeBB devs are NOT asshats and offer me options so I am off to test out the pgsql stuff.
Thanks bunches for all the discussion. Very informative and very much appreciated.
P.S.; any infos on pgsql versions? Docs just say 9.5+ but that was long ago and the need for new maintainer was mentioned so I wonder if post 9.x major revs are "supported"?
-
Pgsql v10 works perfectly for us (although our forum is tiny so it may not be a good test case). @Ben-Lubar however, did a number of performance comparison to mongo-DB, and it looks like pgsql is on par. You can follow his work in tdwtf. Good luck!
-
@JJSagan said in Mongodb or PostgreSQL? Ah, that is the question...:
Pgsql v10 works perfectly for us (although our forum is tiny so it may not be a good test case). @Ben-Lubar however, did a number of performance comparison to mongo-DB, and it looks like pgsql is on par. You can follow his work in tdwtf. Good luck!
Ah, finally a pgsql person weighs in. I figured I'd opt for 11.x and see how it went, in absence of other infos.
tdwtf is pretty slow from my location comparitively. Maybe network issues? pings are not too bad and they're kind of boggy for at least one other as well so I was suspecting else.
I'm not really interested in registering there just so I can use search. Any relevant pointers appreciated.
Thanks bunches o/
-
Not much to learn from my experience, since my #users/day is pathetic.
As for tdwtf, I personally don't believe it would have been faster with Mongo, but I could be wrong. I looked but unfortunately I could not find the exact benchmark chart I was referring to earlier. Nevertheless, see here: https://what.thedailywtf.com/topic/25362/so-about-that-postgresql-thing/8?lang=en-US&page=1Finally, you should consider visiting tdwtf often, the posts there are among the funniest I have ever encountered
-
@JJSagan said in Mongodb or PostgreSQL? Ah, that is the question...:
Not much to learn from my experience, since my #users/day is pathetic.
As for tdwtf, I personally don't believe it would have been faster with Mongo, but I could be wrong. I looked but unfortunately I could not find the exact benchmark chart I was referring to earlier. Nevertheless, see here: https://what.thedailywtf.com/topic/25362/so-about-that-postgresql-thing/8?lang=en-US&page=1Finally, you should consider visiting tdwtf often, the posts there are among the funniest I have ever encountered
Thanks bunches, mon! That link is the kind of stuff I want to digest. I'm on it
-
@gotwf said in Mongodb or PostgreSQL? Ah, that is the question...:
Sure, I could spin up a Linux vm, but I already been there, done that and there are good reasons I prefer SmartOS (yeah, even over fbsd server side).
So SmartOS is your planned NodeBB deployment platform?
-
@JJSagan said in Mongodb or PostgreSQL? Ah, that is the question...:
Not much to learn from my experience, since my #users/day is pathetic.
As for tdwtf, I personally don't believe it would have been faster with Mongo, but I could be wrong. I looked but unfortunately I could not find the exact benchmark chart I was referring to earlier. Nevertheless, see here: https://what.thedailywtf.com/topic/25362/so-about-that-postgresql-thing/8?lang=en-US&page=1Finally, you should consider visiting tdwtf often, the posts there are among the funniest I have ever encountered
Holy cow, I didn't realize that we were busier than them! I figured that they were the busier site. So our MongoDB is handling more traffic. Still crazy busy, though.
-
@scottalanmiller said in Mongodb or PostgreSQL? Ah, that is the question...:
@gotwf said in Mongodb or PostgreSQL? Ah, that is the question...:
Sure, I could spin up a Linux vm, but I already been there, done that and there are good reasons I prefer SmartOS (yeah, even over fbsd server side).
So SmartOS is your planned NodeBB deployment platform?
Yes. Or at least was when I was using Redis for an experimental testing site that is no longer extant. Now I will test under PostgreSQL and see how it goes. Might need to retract to fbsd and mongodb if that does not prove out well.
I am completely off AWS and DO. SmartOS has become my platform of choice, as my SmartOS vm's provide noticeably greater bang for the buck, even if the 'stats' seem like less. Joyent is just more honest about it - and/or maybe zones provide better and/or more consistent resource management. It has been a few years since I tested the competition, however, mayhaps things have changed. Not worth my time to go test again though cuz I am a pretty happy camper where I am
-
@scottalanmiller said in Mongodb or PostgreSQL? Ah, that is the question...:
@JJSagan said in Mongodb or PostgreSQL? Ah, that is the question...:
Not much to learn from my experience, since my #users/day is pathetic.
As for tdwtf, I personally don't believe it would have been faster with Mongo, but I could be wrong. I looked but unfortunately I could not find the exact benchmark chart I was referring to earlier. Nevertheless, see here: https://what.thedailywtf.com/topic/25362/so-about-that-postgresql-thing/8?lang=en-US&page=1Finally, you should consider visiting tdwtf often, the posts there are among the funniest I have ever encountered
Holy cow, I didn't realize that we were busier than them! I figured that they were the busier site. So our MongoDB is handling more traffic. Still crazy busy, though.
Aye... so now... we gots a significantly busier MongoDB based site that is substantially snappier than the only moderate to large PostgreSQL example that I am aware of. Which once again begs the question: Why?
Should be an adventure. I'll keep y'all posted
-
@gotwf said in Mongodb or PostgreSQL? Ah, that is the question...:
Might need to retract to fbsd and mongodb if that does not prove out well.
Can't find a FreeBSD package of MongoDB. Does that have a supported install?
-
@julian said in Mongodb or PostgreSQL? Ah, that is the question...:
too many variables to know for sure. They could be like us and host NodeBB on a potato server
Yeah, our NodeBB platform has some muscle behind it, and CloudFlare in front. That all helps, a lot. Our working set is 100% in RAM with loads of overhead. We always have threads idle. Although we also serve several other communities off of the same machine, it's a shared server.
-
Just an update to note that I've bailed on SmartOS and PostgreSQL due to too many bumps in the road in favor of sticking with a well supported platform. Which pretty much translates to: "what the dev heads are using...".
I use fbsd for our in house stuff that I keep updated via build server and internal lan. When I need cloud scale smartos or linux are the best options. To wit I have not even tried fbsd and pgsql so cannot weigh in there.
Thanks for all the insights. I'll probably be back to bug ya' before too long.
o/
