How to export from Redis to MongoDB? (My database got wiped)


  • Community Rep

    You're going to have to help us out here, there is a lot of variability in how to install various parts of nodebb, being hand wavy and saying "it doesn't work" helps no one.

    • what provider are you on?
    • what versions of nodebb and redis?
    • are you using nginx? Do you have ports open to the db from the outside world?
    • can you give logs from the database / nodebb / system?
      Etc. etc.


  • OVH.
    Ubuntu Linux server.
    NodeBB v0.8.2.
    Redis server v3.0.5.
    No nginx.
    No open ports from outside.

    There are no logs from the crash. It. Just. Stopped. Working. 100% dead, no errors.



  • No Nginx or Apache... I find that hard to beleave. Are you running NodeBB on port 80?


  • Community Rep

    If there are really no logs it makes me think you might have forgotten to turn a config off, something that would have the database written to disk every once in a while. Did you import a community by any chance?



  • @Kowlin said:

    No Nginx or Apache... I find that hard to beleave. Are you running NodeBB on port 80?

    I'm running Apache of course.



  • @BDHarrington7 said:

    If there are really no logs it makes me think you might have forgotten to turn a config off, something that would have the database written to disk every once in a while. Did you import a community by any chance?

    There are logs but nothing useful. They just stop before the crash happened.

    Several months ago I imported messages from SMF but that's it.



  • @Pilvinen I encountered a similar issue as yours. The logs showed no problem at all nor NodeBB ones as it is not the softwares fold.
    This issue is 100% related to a config issue as it showed out for me.

    First of all, are you sure that Redis hat R/W access to the directory /var/lib/redis and its files?
    Yes. -> Post config file.
    No. -> chown -R redis:redis /var/lib/redis



  • @AOKP

    Of course it has write access there, I've been running the forums for many months and I have taken backups along the way every now and then and the dump.rdb has been alive and well.

    When the crash happened -> dump.rdb was suddenly 4 bytes in size instead of the usual ~13 mb.

    Which config do you wish to see? /etc/redis/redis.conf?

    It's 100% default except now after the second crash I switched appendonly to yes, ie. I'm now using appendonly.aof + dump.rdb both.

    Do you still want to see it ... ?



  • @Pilvinen said:

    @AOKP
    Which config do you wish to see? /etc/redis/redis.conf?

    Yes.


  • Global Moderator

    We want to see all the configs!



  • Here is /etc/redis/redis.conf:
    http://pastie.org/private/rehppriwl3yfgcmkk4gtq

    And it would be a lot more productive to say exactly which configs instead of "all of them" since I'm not power user - I don't know what you're talking about.

    Like I said I have default settings except those which I've configured through ACP.

    So tell me the config file names and paths and I'll see what I can pull up.


  • Staff Admin

    Also post the output of free -m, make sure you have enough ram and swap enabled.



  • free -m:

                 total       used       free     shared    buffers     cached
    Mem:         32141      29098       3043       3527        621      11891
    -/+ buffers/cache:      16585      15555
    Swap:         1025         37        988
    


  • @Pilvinen I hope you have a password for Redis...
    Otherwise I haven't found any issue.

    What you can also try is the following:
    service redis-server stop
    apt-get remove redis*
    apt-get purge redis*

    apt-get install redis-server

    So you have a fresh reinstall.



  • @AOKP said:

    @Pilvinen I hope you have a password for Redis...
    Otherwise I haven't found any issue.

    Well, since you can only connect to the database from the local machine I don't really see the point. If someone would compromise my system so far that they have access to the server's account then password protecting the database wouldn't really help at that point.



  • @Pilvinen my config also is bind only to localhost, but I still can access it from the outter world.
    Otherwise change the log level to debug as well.



  • @AOKP said:

    @Pilvinen my config also is bind only to localhost, but I still can access it from the outter world.

    What... ? Why? How? What do you mean? Why would it accept connections from the outside? That makes no sense.



  • I don't really see why it would accept connections from the outside - but since you scared me like that I have now password protected my database...


  • Community Rep

    If there's no firewall it's completely possible


  • GNU/Linux

    Hi @Pilvinen

    I hope you didn't purge redis, already. That could have deleted the log file, as well.

    I'm curious about the contents of two files:

    • /var/log/redis/redis-server.log
    • /path/to/nodebb/log/nodebb.log

    Like all the others, I suspect redis itself to be the source of failure here. NodeBB uses the DB's flush (wipe all data) only in its tests; even there on a test-databse you'd have to set up explicitly.



Suggested Topics

  • 5
  • 2
  • 27
  • 2
  • 6
| |