How to export from Redis to MongoDB? (My database got wiped)

[Charly86]

In my case it's saved every 5 minutes onto disk, so I should have lost only 5 minutes, except if as you says, it saved corrupted db.

297] 08 Nov 02:19:36.116 * Background saving terminated with success
[297] 08 Nov 02:24:37.088 * 10 changes in 300 seconds. Saving...
[297] 08 Nov 02:24:37.091 * Background saving started by pid 4508
[4508] 08 Nov 02:24:37.114 * DB saved on disk
[4508] 08 Nov 02:24:37.115 * RDB: 0 MB of memory used by copy-on-write
[297] 08 Nov 02:24:37.192 * Background saving terminated with success
[297] 08 Nov 02:29:38.091 * 10 changes in 300 seconds. Saving...
[297] 08 Nov 02:29:38.092 * Background saving started by pid 4511
[4511] 08 Nov 02:29:38.110 * DB saved on disk
[4511] 08 Nov 02:29:38.110 * RDB: 0 MB of memory used by copy-on-write
[297] 08 Nov 02:29:38.193 * Background saving terminated with success
[297] 08 Nov 02:34:39.034 * 10 changes in 300 seconds. Saving...
[297] 08 Nov 02:34:39.035 * Background saving started by pid 4514
[4514] 08 Nov 02:34:39.059 * DB saved on disk

[Charly86]

after corruption I connected to redis without issue but database structure was

instead of

[A Former User]

Why don't you try AOF instead of RDB?
It can save every second, without any certain performance loss. The only disadvantage is that it uses more disk space.

[Charly86]

yeah no problem for me using AOF, I need to set it in redis.conf ?

[A Former User]

@Charly86 yep.

[Charly86]

I'll check this thanks

[A Former User]

Another idea would be to configure a cronjob running every X minutes, copying the database.
cp -R /var/lib/redis.rdb /root

[Charly86]

Yeah I've already put a backup into /etc/cron.hourly as a fix

[wellenreiter]

@pitaj My version is this:

web02:~$ redis-server -v
Redis server v=3.0.3 sha=00000000:0 malloc=jemalloc-3.6.0 bits=64 build=e1e08a07153b9ea1

[A Former User]

@wellenreiter are you also a OVH user?

[wellenreiter]

@AOKP not sure what you mean with OVH and I'm not the one who leased the host, but it's a linode server, so I assume it's a virtual machine

[A Former User]

@wellenreiter OVH is an ISP, providing servers.
However, I can only assume that something is wrong with your Redis setup or servers.

Does anyone uses the Redis repo from Chris Lea?

[wellenreiter]

@AOKP said:

@wellenreiter OVH is an ISP, providing servers.
However, I can only assume that something is wrong with your Redis setup or servers.

Then, why occurs the problem to a number of different installations in just a couple of days on systems that have run fine for month before?
Also as mentioned above, my redis configuration is 99.9% the default config

[A Former User]

@wellenreiter as said. I am running Redis since May and never had a problem similar to this.
The whole case is indeed very strange...

[wellenreiter]

It is really strange and if you count all the different installations mentioned here and in the thread 'lost all my data' I doubt, that it is related to the redis configuration

[A Former User]

@wellenreiter but then again, why the issue appears also on different NodeBB versions?
Maybe really give the PPA Repo from Chris Lea a try or switch to MongoDB.

[mikep]

This post is deleted!

[A Former User]

@mikep I used NodeBB 0.7.2, 0.8.x and now 0.9. 16GB DDR4 ECC RAM, Intel Xeon E5-2690v3, RAID10 Intel 750 NVMe.
I even had Redis 2.8.x running (current one is 3.0.5).

By the way, it means NodeBB or something else would have to make a "FLUSHDB" request, except it is a memory error.

[bitspook]

@mikep said:

FYI I have had the same problem. I am using a digital ocean 4GB RAM/2CPU VPS with Ubuntu 14.04, nodebb 0.81 and redis 3.04, so I don't think that it is provider/config related.

mmmm, wait. That is almost exactly our dev forum setup. The only difference is nodebb .9

Lost two days of talk between devs and admins...

[mikep]

This post is deleted!