How to export from Redis to MongoDB? (My database got wiped)
-
I am at the end of my wits right now. I just experienced a second total meltdown of my Redis database. Everything was fine - then suddenly I had a database of the size of 4 bytes, all data is lost and nothing works.
I had to use a backup that was a week old so my users lost probably hundreds of messages and private messages.
In my opinion this should never ever happen with any forum software - and never has happened to me except with Nodebb, twice.
I am willing to give Nodebb only one more try before switching to another forum software forever.
HOW do I export my Redis database to Mongodb? I'm not using that unreliable piece of **** a second longer that I have to.
Help?
-
never has happened to me except with Nodebb, twice
Are you referring specifically to a Redis meltdown, or just a database fault in general? If there is something wrong with the redis driver, that needs to be addressed.
-
Not knowing the full story, I'm inclined to blame a custom installation/configuration of Redis with persistence disabled or unintentionally broken.
Either way, Redis to Mongo conversions are something we charge a nominal fee for. Perhaps you'd be interested in having us do it for you?
-
Default installation. Redis database.
And I don't know why it does this but this is already second time that this happens. No error AFAIK.
After this happened second time -> I made a bash backup script that now takes backups every hour with 24 day history. So that when this happens a third time I won't lose more than 1 hour worth of data. -
Yeah, thanks. Of course I save to a different file each time. I know how to make bash scripts.
What I don't know is how to prevent NodeBB + Redis database from crashing and wiping itself without no reason at all, but then again that's probably your job - and so far it's not going very well is it?
-
For all we know it can be your host, your configuration and whatnot destroying databases. As @BDHarrington7 said, you're the only one who got these problems. I've broken instances of NodeBB (for the sake of this testing) many times and I never had it wipe a database. I surguest you take a good look at all your logs. The way you run NodeBB, the way you take backups. Etc etc. Even if the software is at fault here. Something about the way you're running it is causing it
-
Did you build Redis form source? It happened to me once when I did that, which probably indicates the problem came from something I did (or didn't do) during the build process.
-
You're going to have to help us out here, there is a lot of variability in how to install various parts of nodebb, being hand wavy and saying "it doesn't work" helps no one.
- what provider are you on?
- what versions of nodebb and redis?
- are you using nginx? Do you have ports open to the db from the outside world?
- can you give logs from the database / nodebb / system?
Etc. etc.
-
@BDHarrington7 said:
If there are really no logs it makes me think you might have forgotten to turn a config off, something that would have the database written to disk every once in a while. Did you import a community by any chance?
There are logs but nothing useful. They just stop before the crash happened.
Several months ago I imported messages from SMF but that's it.
-
@Pilvinen I encountered a similar issue as yours. The logs showed no problem at all nor NodeBB ones as it is not the softwares fold.
This issue is 100% related to a config issue as it showed out for me.First of all, are you sure that Redis hat R/W access to the directory /var/lib/redis and its files?
Yes. -> Post config file.
No. -> chown -R redis:redis /var/lib/redis -
Of course it has write access there, I've been running the forums for many months and I have taken backups along the way every now and then and the dump.rdb has been alive and well.
When the crash happened -> dump.rdb was suddenly 4 bytes in size instead of the usual ~13 mb.
Which config do you wish to see? /etc/redis/redis.conf?
It's 100% default except now after the second crash I switched appendonly to yes, ie. I'm now using appendonly.aof + dump.rdb both.
Do you still want to see it ... ?
-
We want to see all the configs!
