@psychobunny said:
Hmm, can you get a stack trace on that error?
Nope - nothing.
@TheBronx
Similar behavior for mine. But seems not the same reason, cause the failed requests are fast enough while there still exist a long request that succeed.
It's NodeBB 1.1.0 and socket.io 1.4.8
digital ocean 8G 4core machine
Followed by my nginx config:
upstream io_nodes {
ip_hash;
server 127.0.0.1:4567;
server 127.0.0.1:4568;
server 127.0.0.1:4569;
server 127.0.0.1:4570;
}
server {
listen 80;
server_name my.server.name;
return 301 https://$server_name$request_uri;
}
server {
# listen on ssl, deliver with speedy if possible
listen 443 ssl;
server_name my.server.name;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $http_host;
proxy_set_header X-NginX-Proxy true;
proxy_redirect off;
gzip on;
gzip_min_length 1000;
gzip_proxied off;
gzip_types text/plain application/xml application/x-javascript text/css application/json;
# Socket.IO Support
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
ssl on;
# change this path!
ssl_certificate /etc/nginx/ssl/ssl.crt;
# change this path!
ssl_certificate_key /etc/nginx/ssl/server.key;
# enables all versions of TLS, but not SSLv2 or 3 which are weak and now deprecated.
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
# disables all weak ciphers
ssl_ciphers 'AES128+EECDH:AES128+EDH';
ssl_prefer_server_ciphers on;
location @nodebb {
# limit_req zone=allips burst=6 nodelay;
proxy_pass http://io_nodes;
}
location ~ ^/(images|language|sounds|templates|uploads|vendor|src\/modules|src\/client|nodebb\.min\.js|stylesheet\.css|admin\.css|js-enabled\.css|favicon\.ico) {
root /home/lcnodebb/nodebb/public/;
try_files $uri $uri/ @nodebb;
}
location / {
access_log /var/log/nginx/access_nodebb.log;
error_log /var/log/nginx/error_nodebb.log;
proxy_pass http://io_nodes;
}
}
I do agree you very much that the polling requests are causing an enormous load on the server. My machine is powerful enough holding 65000+user and 140000+ posts.
But the load is heavy at peak time, and if I restart nodebb, it can not even recover until the peak was passed.
@administrators
Any idea on this? I was obsessed with this for long...
I found that the ESTABLISHED connections are too large. Does that count for this issue?
No matter peak time or valley time, this number almost seems the same.
user@discuss:~$ netstat -n | grep -c ESTABLISHED
14003
and the ulimit:
user@discuss:~$ ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 63711
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 10000
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 63711
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited