Need Mongo to Postgres converter for NodeBB 2.x

[taaliman 0]

Hello, everyone!
Faced with a problem when mongo can not cope with the load and the forum constantly gives an error 504. At the same time mongo itself hugely loads the processor. I would like to switch to postgres and then optimize the database. Can someone help and write a converter from mongo to postgres?

[gotwf]

@taaliman-0 said in Need Mongo to Postgres converter for NodeBB 2.x:

mongo can not cope with the load

Are you sure about this? Mongodb scales horizontally very well. As does NodeBB. And nginx. Ditto Redis if you are employing for session management.

Pray tell what does your provisioning and configuration look like? Could be interesting to do a little tuning.

Not to dissuade you from moving to PostgreSQL, if, for no other reason than licensing freedom considerations. But from pure performance aspects, I would question whether postgres will outperform mongodb without first doing some real time load testing.

[volanar]

@gotwf postgres requires less RAM, less disk space and better performance than mongodb. At least 2 times for all parameters.
https://www.enterprisedb.com/news/new-benchmarks-show-postgres-dominating-mongodb-varied-workloads

[taaliman 0]

@gotwf The configuration is as productive as possible. The forum runs on 6 dedicated servers with mongo replication, redis cluster and balancer at the nginx level. But at peak loads, mongo just does not leave a chance and even there were cases when mongo-service ended with an error.

In addition, I noticed that chats are very slow and I would like to optimize them through separate tables in the database. Chats are the weakest part in NodeBB right now. And I'm surprised and a bit frustrated that there is no way to convert from one database to another. I personally like Postgres much more than Mongo because I have more experience with Postgres.

That's why I'm even ready to financially support development of this tool, because I think that such functionality should be part of NodeBB core.

[volanar]

@taaliman-0 I agree. Chat is the weakest link. Discourse creates a chat plugin tightly integrated into the forum with many features.
https://meta.discourse.org/t/introducing-discourse-chat-beta/210734
Nodebb has lagged behind in this direction and is unlikely to ever catch up with discourse. The only way out, in my opinion, is to move chat from the core to a separate plugin and develop it separately, including integration with rocket chat in the settings with the ability to transfer data. It is impossible to develop a modern chat with a lot of functionality without a large development team

[gotwf]

@taaliman-0 said in Need Mongo to Postgres converter for NodeBB 2.x:

The forum runs on 6 dedicated servers with mongo replication, redis cluster and balancer at the nginx level.

Seems like you've ticked most of the boxes. I wonder, however, if you have explored mongodb sharding?

Sharding — MongoDB Manual

(www.mongodb.com)

I prefer PostgreSQL. Hence, my latest putzing about is with postgres. And MongoDB's licensing games are a concern. But I don't think it sucks as bad as being portrayed here. Especially when you've not implemented sharding, which is what I was referencing here:

@gotwf said in Need Mongo to Postgres converter for NodeBB 2.x:

Mongodb scales horizontally very well.

@volanar Uh, yeah, I now recall we discussed this previously. Not consistent with my last round of testing, which has admittedly been some while.

YMMV. Have fun!