Lost all my data ? Help please


  • @julian @baris @psychobunny ,

    you maybe want to consider now SSDB support? :bowtie:


  • @AOKP
    after loosing several days of posts in the error, I set up a database backup following this article: http://redis4you.com/articles.php?id=010


  • @wellenreiter said:

    @AOKP
    after loosing several days of posts in the error, I set up a database backup following this article: http://redis4you.com/articles.php?id=010

    Mine is similar to this. Except I was too lazy to setup a crontab for X minutes. I simply made a hourly one.


  • @AOKP said:

    Mine is similar to this. Except I was too lazy to setup a crontab for X minutes. I simply made a hourly one.

    Did the same πŸ˜‰


  • Is someone now why this happen ?!! Is it can be related to the hosting provider Digitalocean ?


  • The host system of my installation is not running on digital ocean, as far as I know.
    The provider is called linode??

    Sorry, but I'm not the one leasing the host or dealing with the provider at all


  • Same issue here on a DigitalOcean hosting. The database has first been wiped out while running v0.8.2, then again last night while running v0.9.0.

  • Gamers

    Though it is likely unrelated, lots of Digital Ocean people involved πŸ˜•


  • What's amazing is that happening to several people in the same days interval on different redis version and nodeBB version without touching anything from days !!!

    I've backup up also every hour now !!!

    crontab - e then add

    # 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/
    #
    # For more information see the manual pages of crontab(5) and cron(8)
    #
    # m h  dom mon dow   command
    MAILTO=""
    13 * * * *  /root/redis-backup.sh
    

    And /root/redis-backup.sh (grabbed from Internet, don't remember where and modified to fit my needs).

    dont forget to chmod ug+x /root/redis-backup.sh

    #!/bin/bash
    #
    ## redis backup script
    ## usage
    ## redis-backup.sh port backup.dir
    
    port=${1:-6379}
    backup_dir=${2:-"/var/lib/redis/backup"}
    
    cli="/usr/bin/redis-cli -p $port"
    rdb="/var/lib/redis/dump.rdb"
    
    test -f $rdb || {
      echo "[$port] No RDB Found" ; exit 1
    }
    test -d $backup_dir || {
      echo "[$port] Create backup directory $backup_dir" && mkdir -p $backup_dir
    }
    
    # perform a bgsave before copy
    echo bgsave | $cli
    echo "[$port] waiting for 5 seconds..."
    sleep 5
    try=10
    while [ $try -gt 0 ] ; do
      ## redis-cli output dos format line feed '\r\n', remove '\r'
      bg=$(echo 'info Persistence' | $cli | awk -F: '/rdb_bgsave_in_progress/{sub(/\r/, "", $0); print $2}')
      ok=$(echo 'info Persistence' | $cli | awk -F: '/rdb_last_bgsave_status/{sub(/\r/, "", $0); print $2}')
      if [ "$bg" = "0" ] && [ "$ok" = "ok" ] ; then
        dst="$backup_dir/$port-dump.$(date +%Y%m%d%H%M).rdb"
        cp $rdb $dst
        if [ $? = 0 ] ; then
          echo "[$port] redis rdb $rdb copied to $dst."
    
          # delete rdb created 30 days ago
          cd $backup_dir
          find . \( -name "$port-dump*" \) -mtime +30 -exec rm -f {} \;
          exit 0
        else
          echo "[$port] >> Failed to copy $rdb to $dst!"
        fi
      fi
      try=$((try - 1))
      echo "[$port] redis maybe busy, waiting and retry in 5s..."
      sleep 5
    done
    
    

    If it can help, it can be better and not in root folder, done in a hurry, you know what I mean πŸ™‚


  • Is someone here can explain me where to find if I have a backup.. How can I just lose all my data without a reason ?? Please someone explain me what's could have happened to my website ? I lose a nine months of hard work !!!

    What's command I have to execute to try restoring my datas ?


  • @bitspook I don't want to throw out blind judgement here... Since I never used Digital Ocean. I've always used my own full scale dedicated. But does someone experience issue's with MongoDB and Digital Ocean? And like many people over NodeBB said, keep making backups.
    Edit: I'm also wondering on what scale database wipes occur. They seem to be pretty rare.

  • Gamers

    @Kowlin not that it couldnt be Digital Ocean; just that the correlation is anecdotal as of yet. πŸ˜‰


  • @bitspook Who knows, maby its a blind curse πŸ˜‰. Anyways is someone running more Node.JS apps or Redis databases on there servers? Did those get wiped at one point or another?

  • GNU/Linux Admin

    If you had Redis persisting to disk, it would be at /var/lib/redis.

    It sounds like someone (or some script, rather) is going around trying to connect to port 6379, and if successful, flushing the database.

    Better use AUTH and have good backups!!

  • GNU/Linux Admin

    @kacemlight This seems to be the case. I am able to connect to the redis server at nodejsworld.com:

    $ redis-cli -h nodejsworld.com
    nodejsworld.com:6379> keys *
    1) "sess:0d5qvchIPxvQ-lsGNTJkWL7s1SQBrd8v"
    2) "ip:recent"
    3) "schemaDate"
    

    Please take a look at this topic: http://redis.io/topics/security.

    I realise this is a very rough time for you, @kacemlight, as you've lost your site data. I have every confidence that you will be more careful in the future and utilise backups in addition to setting up authentication for your database.


  • @julian Always a posibility too be honest. I'll have some fun opening my Redis port tonight to my server. Not like I have critical data on there anyways.

  • GNU/Linux Admin

    @Kowlin I've found the easiest way is to just use ufw. If you only connect from one PC, then disallow requests from all IPs except your own, and set the default behaviour to DENY.


  • Well am I the only one a bit troubled about the fact that the default Redis security is so easy to avoid. I opened my Redis port from my router and connected to it without any single problem...

  • Community Rep

    @kacemlight said:

    Is someone here can explain me where to find if I have a backup.. How can I just lose all my data without a reason ?? Please someone explain me what's could have happened to my website ? I lose a nine months of hard work !!!

    What's command I have to execute to try restoring my datas ?

    Did you take a backup of any sort?

  • Gamers

    @julian @Kowlin Redis is that bad? Did not know that. Planned on migrating to Mongodb before production anyway πŸ˜„

Suggested Topics

| |