Redis useful info

General Discussion
  • Being new to Redis myself, I need to start reading up more on it especially with regards to the Redis database being in memory and what that means for database persistence and reliability in event of failure.

    So this thread just serves as a collection point for useful Redis info and links that I find during my reading that might help others.

    Redis Quickstart

    Redis database persistence and reliability

    Redis persistence
    You can learn how Redis persisence works in this page, however what is important to understand for a quick start is that by default, if you start Redis with the default configuration, Redis will spontaneously save the dataset only from time to time (for instance after at least five minutes if you have at least 100 changes in your data), so if you want your database to persist and be reloaded after a restart make sure to call the SAVE command manually every time you want to force a data set snapshot. Otherwise make sure to shutdown the database using the SHUTDOWN command:
    $ redis-cli shutdown
    This way Redis will make sure to save the data on disk before quitting. Reading the persistence page is strongly suggested in order to better understand how Redis persistence works.

    Looks like for Redis 2.8.6 redis.conf AOF is disabeld by default and the default RDB snapshot settings in redis.conf are as follows

    checking my local test NodeBB with Redis 2.8.6

        ls -lah /var/lib/redis;date
        total 20K
        drwxr-xr-x   2 redis root  4.0K Feb 27 08:30 .
        drwxr-xr-x. 23 root  root  4.0K Feb 25 19:34 ..
        -rw-r--r--   1 redis redis 8.5K Feb 27 08:30 dump.rdb
        Thu Feb 27 08:30:54 EST 2014

    from redis.conf it looks like if you have a low activity NodeBB forum using Redis database, the snapshots are written to disk every 5-15mins. If you have a very active forum with >10,000 key changes then it will write to disk the snapshot every 60 seconds or 1 min. In a way still better than MySQL out of the box as unless you specifically setup a MySQL backup policy/plan, you don't get any backups at all !

        ################################ SNAPSHOTTING  ################################
        # Save the DB on disk:
        #   save <seconds> <changes>
        #   Will save the DB if both the given number of seconds and the given
        #   number of write operations against the DB occurred.
        #   In the example below the behaviour will be to save:
        #   after 900 sec (15 min) if at least 1 key changed
        #   after 300 sec (5 min) if at least 10 keys changed
        #   after 60 sec if at least 10000 keys changed
        #   Note: you can disable saving at all commenting all the "save" lines.
        #   It is also possible to remove all the previously configured save
        #   points by adding a save directive with a single empty string argument
        #   like in the following example:
        #   save ""
        save 900 1
        save 300 10
        save 60 10000
        # By default Redis will stop accepting writes if RDB snapshots are enabled
        # (at least one save point) and the latest background save failed.
        # This will make the user aware (in a hard way) that data is not persisting
        # on disk properly, otherwise chances are that no one will notice and some
        # disaster will happen.
        # If the background saving process will start working again Redis will
        # automatically allow writes again.
        # However if you have setup your proper monitoring of the Redis server
        # and persistence, you may want to disable this feature so that Redis will
        # continue to work as usual even if there are problems with disk,
        # permissions, and so forth.
        stop-writes-on-bgsave-error yes
        # Compress string objects using LZF when dump .rdb databases?
        # For default that's set to 'yes' as it's almost always a win.
        # If you want to save some CPU in the saving child set it to 'no' but
        # the dataset will likely be bigger if you have compressible values or keys.
        rdbcompression yes
        # Since version 5 of RDB a CRC64 checksum is placed at the end of the file.
        # This makes the format more resistant to corruption but there is a performance
        # hit to pay (around 10%) when saving and loading RDB files, so you can disable it
        # for maximum performances.
        # RDB files created with checksum disabled have a checksum of zero that will
        # tell the loading code to skip the check.
        rdbchecksum yes
        # The filename where to dump the DB
        dbfilename dump.rdb
        # The working directory.
        # The DB will be written inside this directory, with the filename specified
        # above using the 'dbfilename' configuration directive.
        # The Append Only File will also be created inside this directory.
        # Note that you must specify a directory here, not a file name.
        dir /var/lib/redis/
  • AOF redis.conf settings

        ############################## APPEND ONLY MODE ###############################
        # By default Redis asynchronously dumps the dataset on disk. This mode is
        # good enough in many applications, but an issue with the Redis process or
        # a power outage may result into a few minutes of writes lost (depending on
        # the configured save points).
        # The Append Only File is an alternative persistence mode that provides
        # much better durability. For instance using the default data fsync policy
        # (see later in the config file) Redis can lose just one second of writes in a
        # dramatic event like a server power outage, or a single write if something
        # wrong with the Redis process itself happens, but the operating system is
        # still running correctly.
        # AOF and RDB persistence can be enabled at the same time without problems.
        # If the AOF is enabled on startup Redis will load the AOF, that is the file
        # with the better durability guarantees.
        # Please check for more information.
        appendonly no
        # The name of the append only file (default: "appendonly.aof")
        appendfilename "appendonly.aof"
        # The fsync() call tells the Operating System to actually write data on disk
        # instead to wait for more data in the output buffer. Some OS will really flush 
        # data on disk, some other OS will just try to do it ASAP.
        # Redis supports three different modes:
        # no: don't fsync, just let the OS flush the data when it wants. Faster.
        # always: fsync after every write to the append only log . Slow, Safest.
        # everysec: fsync only one time every second. Compromise.
        # The default is "everysec", as that's usually the right compromise between
        # speed and data safety. It's up to you to understand if you can relax this to
        # "no" that will let the operating system flush the output buffer when
        # it wants, for better performances (but if you can live with the idea of
        # some data loss consider the default persistence mode that's snapshotting),
        # or on the contrary, use "always" that's very slow but a bit safer than
        # everysec.
        # More details please check the following article:
        # If unsure, use "everysec".
        # appendfsync always
        appendfsync everysec
        # appendfsync no
        # When the AOF fsync policy is set to always or everysec, and a background
        # saving process (a background save or AOF log background rewriting) is
        # performing a lot of I/O against the disk, in some Linux configurations
        # Redis may block too long on the fsync() call. Note that there is no fix for
        # this currently, as even performing fsync in a different thread will block
        # our synchronous write(2) call.
        # In order to mitigate this problem it's possible to use the following option
        # that will prevent fsync() from being called in the main process while a
        # BGSAVE or BGREWRITEAOF is in progress.
        # This means that while another child is saving, the durability of Redis is
        # the same as "appendfsync none". In practical terms, this means that it is
        # possible to lose up to 30 seconds of log in the worst scenario (with the
        # default Linux settings).
        # If you have latency problems turn this to "yes". Otherwise leave it as
        # "no" that is the safest pick from the point of view of durability.
        no-appendfsync-on-rewrite no
        # Automatic rewrite of the append only file.
        # Redis is able to automatically rewrite the log file implicitly calling
        # BGREWRITEAOF when the AOF log size grows by the specified percentage.
        # This is how it works: Redis remembers the size of the AOF file after the
        # latest rewrite (if no rewrite has happened since the restart, the size of
        # the AOF at startup is used).
        # This base size is compared to the current size. If the current size is
        # bigger than the specified percentage, the rewrite is triggered. Also
        # you need to specify a minimal size for the AOF file to be rewritten, this
        # is useful to avoid rewriting the AOF file even if the percentage increase
        # is reached but it is still pretty small.
        # Specify a percentage of zero in order to disable the automatic AOF
        # rewrite feature.
        auto-aof-rewrite-percentage 100
        auto-aof-rewrite-min-size 64mb
  • Thought I'd share a shell script I wrote, to grab all relevant redis server memory and usage info so I can analyse the stats and check for memory swapping issues etc.

    I uploaded the script at

    The first reply in the comments there also has instructions on installing including sample output of the info and stats gathered.


    redis-cli 2.8.9
    redis-cli info
    # Server
    os:Linux 2.6.32-431.el6.x86_64 x86_64
    # Clients
    # Memory
    # Persistence
    # Stats
    # Replication
    # CPU
    # Keyspace
    redis-cli --intrinsic-latency 10
    Max latency so far: 4 microseconds.
    Max latency so far: 6 microseconds.
    Max latency so far: 13 microseconds.
    Max latency so far: 16 microseconds.
    Max latency so far: 32 microseconds.
    3214753 total runs (avg 3 microseconds per run).
    Worst run took 10.67x times the avarege.
    vmstat 1 10
    procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu-----
     r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
     0  0      0 15453032 140484 407852    0    0     0     0   16   24  4  5 91  0  0
     0  0      0 15453032 140484 407852    0    0     0     0  188  150  0  0 100  0  0
     0  0      0 15453032 140484 407852    0    0     0     0  126  138  0  0 100  0  0
     0  0      0 15453032 140484 407852    0    0     0     2  151  188  0  0 100  0  0
     1  0      0 15453032 140484 407852    0    0     0     0  129  146  0  0 100  0  0
     0  0      0 15453032 140484 407852    0    0     0     0  132  154  0  0 100  0  0
     0  0      0 15453032 140484 407852    0    0     0     0  125  142  0  0 100  0  0
     0  0      0 15453032 140484 407852    0    0     0     0  136  150  0  0 100  0  0
     0  0      0 15453032 140484 407852    0    0     0     0  124  138  0  0 100  0  0
     0  0      0 15453040 140484 407852    0    0     0     0  137  164  0  0 100  0  0
    smem -kU redis
      PID User     Command                         Swap      USS      PSS      RSS 
    31142 redis    /usr/sbin/redis-server 127.        0    19.1M    19.2M    19.7M 
    smem -kmU redis
    Map                                       PIDs   AVGPSS      PSS 
    /usr/lib/locale/locale-archive               1        0        0 
    [vdso]                                       1        0        0 
    [vsyscall]                                   1        0        0 
    /lib64/                         1     8.0K     8.0K 
    /lib64/                          1     8.0K     8.0K 
    /lib64/                            1     9.0K     9.0K 
    /lib64/                    1    12.0K    12.0K 
    [stack]                                      1    16.0K    16.0K 
    /lib64/                          1    49.0K    49.0K 
    [heap]                                       1    52.0K    52.0K 
    /usr/sbin/redis-server                       1   476.0K   476.0K 
    <anonymous>                                  1    18.5M    18.5M 
    smem -tpk
      PID User     Command                         Swap      USS      PSS      RSS 
     2628 root     /sbin/mingetty /dev/tty4           0    80.0K    99.0K   576.0K 
     2622 root     /sbin/mingetty /dev/tty1           0    84.0K   103.0K   580.0K 
     2624 root     /sbin/mingetty /dev/tty2           0    84.0K   103.0K   580.0K 
     2626 root     /sbin/mingetty /dev/tty3           0    84.0K   103.0K   580.0K 
     2630 root     /sbin/mingetty /dev/tty5           0    84.0K   103.0K   580.0K 
     2632 root     /sbin/mingetty /dev/tty6           0    84.0K   103.0K   580.0K 
     1295 nsd      /usr/local/sbin/nsd -c /etc        0    92.0K   234.0K   960.0K 
     1341 root     mdadm --monitor --scan -f -        0   284.0K   297.0K   628.0K 
     1298 nsd      /usr/local/sbin/nsd -c /etc        0   184.0K   302.0K   708.0K 
     1244 root     auditd                             0   352.0K   372.0K   860.0K 
      608 root     /sbin/udevd -d                     0   136.0K   374.0K     1.1M 
     2634 root     /sbin/udevd -d                     0   116.0K   378.0K     1.1M 
     2635 root     /sbin/udevd -d                     0   116.0K   378.0K     1.1M 
    22942 root     nginx: master process /usr/        0   176.0K   407.0K     1.6M 
    31771 root     /bin/bash /root/tools/redis        0   200.0K   429.0K     1.4M 
     1361 dbus     dbus-daemon --system               0   504.0K   519.0K   868.0K 
     1427 root     /bin/sh /usr/bin/mysqld_saf        0   320.0K   558.0K     1.6M 
     2609 root     crond                              0   648.0K   684.0K     1.2M 
     2498 nobody   /usr/local/bin/memcached -d        0   752.0K   771.0K     1.1M 
        1 root     /sbin/init                         0   736.0K   779.0K     1.5M 
     1405 root     /usr/sbin/sshd                     0   716.0K   807.0K     1.2M 
     1260 root     /sbin/rsyslogd -i /var/run/        0     1.1M     1.1M     1.7M 
     1413 ntp      ntpd -u ntp:ntp -p /var/run        0   960.0K     1.1M     2.1M 
     2586 root     /usr/libexec/postfix/master        0     1.1M     1.2M     2.1M 
    22944 nginx    nginx: worker process              0     1.5M     1.8M     3.4M 
    31413 postfix  pickup -l -t unix -u               0     1.1M     1.8M     3.8M 
    22943 nginx    nginx: worker process              0     1.5M     1.8M     3.4M 
    22946 nginx    nginx: worker process              0     1.5M     1.8M     3.5M 
    22945 nginx    nginx: worker process              0     1.5M     1.8M     3.5M 
     2588 postfix  qmgr -l -t unix -u                 0     1.3M     1.9M     4.0M 
    29348 root     sshd: root@pts/0                   0     1.9M     2.4M     4.1M 
    29351 root     -bash                              0     2.6M     2.9M     4.0M 
     2487 root     /usr/sbin/haveged -w 3072 -        0     3.1M     3.1M     3.5M 
     1297 nsd      /usr/local/sbin/nsd -c /etc        0     4.2M     4.3M     4.8M 
    22960 root     php-fpm: master process (/u        0     5.3M     5.4M     6.0M 
    31796 root     python /usr/bin/smem -tpk          0     5.9M     6.0M     6.7M 
     1335 root     /sbin/mdmon --takeover md0         0    11.4M    11.5M    12.4M 
    26528 root     lfd - sleepin                      0    14.0M    14.0M    14.6M 
    31142 redis    /usr/sbin/redis-server 127.        0    19.1M    19.2M    19.7M 
     2474 mysql    /usr/sbin/mysqld --basedir=        0    38.2M    38.5M    40.1M 
       40 9                                           0   122.9M   129.3M   163.6M 
    free -mlt
                 total       used       free     shared    buffers     cached
    Mem:         15941        851      15090          0        137        398
    Low:         15941        851      15090
    High:            0          0          0
    -/+ buffers/cache:        315      15625
    Swap:         8039          0       8039
    Total:       23981        851      23130
  • Thanks for this 🙂 somehow managed to miss this topic. If you like, feel free to add what you can to our documentation!

  • @psychobunny said:

    Thanks for this 🙂 somehow managed to miss this topic. If you like, feel free to add what you can to our documentation!

    You're welcome 🙂

    oh we can edit the docs too ? might be best if you added it for now 🙂

Suggested Topics

  • 0 Votes
    4 Posts

    @jeffalo yes you can connect to NodeBB more directly and use session-sharing instead.

  • MongoDB over Redis?

    General Discussion
    0 Votes
    1 Posts

    Every now and then when I browse the forum I see that most people are using MongoDB instead of Redis.
    However, as Redis is faster (at least from what I have heard) I wonder why the majority is going for Mongo?

    Is there any benefit?

    I am well aware that Redis needs RAM, however as of now prices for RAM are hilariously low compared to a year or two ago.

  • 0 Votes
    1 Posts

    Solve this with:

    # chmod o-rwx /var/lib/redis # chmod o-rwx /var/lib/mongo

    If you are the only user with access to the server it's not a big problem. However, it's better to make it inaccessible by other users just in case a cracker get access with non-root user account.

  • 0 Votes
    3 Posts

    Thanks.. As an emergency I just removed the topic call (I suspected it was that) every post in the recent list went from a few hundred views to 1k+ in half a day. There aren't enough employees in my department for that many views :). I was only using it to show up votes.. I would have just used the rss feed if I could have but it's hardly includes any information.

    Subtracting from the view counts sounds doable..

    Thanks again.

  • 0 Votes
    6 Posts

    I know this is old but it came up when I was looking for an answer, so for those that come after.....

    Redis is already on c9 and its old.
    Following is some quick instructions that get it working but please understand i REALLY DONT know what Im doing 😉
    The instructions are base off...

    Remove the old redis stuff
    Do you need to do this? idea 😉

    $ sudo service redis-server stop $ sudo dpkg -P redis-server $ sudo dpkg -P redis-tools

    Download and make the new Redis 3.0.x

    paez@nodebb:~/workspace $ mkdir redis $ cd redis wget tar xzf redis-stable.tar.gz cd redis-stable make sudo make install

    During the install the default path for where to put the executable s was wrong and I think it was /usr/bin/
    Also, select port 16379, because the old redis is still running and Im still not sure how to turn it off properly. But your new one will work fine if you set nodeBB to the new one. I did get rid of it once, but not exactly sure how.

    cd utils sudo ./

    And now you should have a working redis on
    Still have the old one running, as I dont know how to get rid of it properly yet.

    I tested this with the latest NodeBB and was able to get it to work except I get this error in the console....

    WebSocket connection to 'ws://' failed: Error during WebSocket handshake: Unexpected response code: 400

    ...and a whole bunch of....
    [deprecated] variables.set is deprecated, please use
    ...dont know why and am not likely to find out as Im going to be running it off Openshift, but this hopefully gets someone a little closer....and it does seem to work good enough for theme mucking which is what I wanted the c9 one for.