Guiri's Guide to Fixing Redis Problems
I'm including in this post the various fixes and optimizations that have made Redis stable on an RHEL 6.5 system with 2GB of memory. I currently run NodeBB/Redis behind a Varnish Cache + nginx combo. Switching to nginx was easy and significantly cut my memory usage.
I am merely combining the advice given to me by other experts. The credit is shared by @julian, @psychobunny, @eva2000, @a_5mith , and others.
/etc/sysctl.conf
net.core.somaxconn = 1024vm.overcommit_memory = 1
/etc/redis.conf
Close dead and idle peers to free up connections. This was my biggest problem that would eventually cause NodeBB to fail to connect and elicit the _csrf error. On RHEL/CentOS, Redis does not by default seem to be configured to close idle connections. Thus, whenever your forum will crash is predictable at a linear rate, where the beta is your avg. number of visitors.
timeout 120tcp-keepalive 60
/etc/security/limits.conf
Increase the maximum number of open files for Redis and your web server.
redis soft nofile 65536
redis hard nofile 65536
nginx soft nofile 262144
nginx hard no file 262144
Using Varnish with nginx
--------------------------------
There is some debate about whether it is more efficient to use nginx's caching capabilities. The additionally memory overhead of using Varnish with nginx is minimal and the scaling performance on my old Core2Duo Mac Mini is incredible as measured by `ab` and `httperf`. My limited understanding is that it recycles old connections, which in turn reduces the load on NodeBB and Redis. Enabling this doubled the time it took to crash Redis, but the eventual fix was modifying `redis.conf` to time out old connections.
**Varnish**
The default [Varnish instructions](https://docs.nodebb.org/en/latest/configuring/proxies/varnish.html) work well, but still need to be updated to 4.0 standards. Thus, since RHEL/CentOS has an older version, install the 3.0 branch from [here](http://repo.varnish-cache.org/redhat/varnish-3.0/el6/), as the syntax is compatible. Varnish outputs on port `6081`, so we just need to slightly modify the nginx default.conf to make it work.
Enable Varnish to start on boot: `sudo chkconfig varnish on`
**nginx**
Once again, you'll first need to install the latest nginx from [their repo](http://wiki.nginx.org/Install). The only line I believe you need to modify at this point is the proxy_pass port:
server {
listen 80;
server_name domain.com *.domain.com;
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header X-NginX-Proxy true;
proxy_pass http://127.0.0.1:6081/;
#proxy_pass http://127.0.0.1:4567/;
proxy_redirect off;
# Socket.IO Support
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
}
Last, make sure you enable nginx to start on boot: `sudo chkconfig nginx on`