400 and 522 errors since enabling SSL via Nginx proxy

[KingCat]

So everything was working great but due to the nature of my site and it's users, SSL is required, especially since I'll be using SSO exclusively for login. Once I enabled SSL I started receiving both 400 and 522 errors on a certain amount of the socket.io polls. I'm not certain what the percentage is of failed polling attempts but it's not 100% because the site still works and some data still makes it through.

Here's a screenshot of the errors in my safari console (yes, it happens in chrome and firefox too)

And here's my nginx vhost config

server {
    listen 80;

    server_name forum.nzb.cat;

    return 301    https://forum.nzb.cat$request_uri;

}
server {
    listen 443 ssl;

    server_name forum.nzb.cat;

    ssl_certificate /etc/nginx/ssl/forum.nzb.cat.crt;
    ssl_certificate_key /etc/nginx/ssl/nzbcat.key;

    location / {
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_set_header X-NginX-Proxy true;

        proxy_pass http://127.0.0.1:4567/;
        proxy_redirect off;

        # Socket.IO Support
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}

And here's the error in the nginx error.log. I googled and googled and did everything the stack overflow people suggested to no avail:

2015/04/12 10:39:03 [error] 17037#0: *146836 upstream prematurely closed connection while reading response header from upstream, client: xxx.xxx.xxx.xxx, server: forum.nzb.cat, request: "GET /socket.io/?EIO=3&transport=polling&t=1428849532722-383&sid=ECw2nTDnOQgEjGcdAAKp HTTP/1.1", upstream: "http://127.0.0.1:4567/socket.io/?EIO=3&transport=polling&t=1428849532722-383&sid=ECw2nTDnOQgEjGcdAAKp", host: "forum.nzb.cat", referrer: "https://forum.nzb.cat/"

I'm at my wit's end here so any help would be appreciated. I didn't bother commenting out any of the URL's as it's a public site.

[Moritz Friedrich]

I had a hard time too while configuring. Try this one:

### redirects http requests to https
server {
    listen 80;
    server_name forum.example.org;
    
    return 301 https://forum.example.org$request_uri;
}

### the https server
server {
    # listen on ssl, deliver with speedy if possible
    listen 443 ssl spdy;

    server_name forum.example.org;

    ssl on;
    
    # change this path!
    ssl_certificate /path/to/cert/bundle.crt;

    # change this path!
    ssl_certificate_key /path/to/cert/forum.example.org.key;
    
    # enables all versions of TLS, but not SSLv2 or 3 which are weak and now deprecated.
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    
    # disables all weak ciphers
    ssl_ciphers 'AES128+EECDH:AES128+EDH';
    
    ssl_prefer_server_ciphers on;

    location / {
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_set_header X-NginX-Proxy true;

        proxy_pass http://127.0.0.1:4567/;
        proxy_redirect off;

        # Socket.IO Support
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}

[Moritz Friedrich]

Alternatively, my working configuration for clustering enabled + SSL:

# redirects http requests to https
server {
  listen 80;
  
	# change this!
  server_name example.com;

	# change this!
  return 301 https://example.com$request_uri;
}

# create multiple instances for scaling
upstream io_nodes {
    ip_hash;
    server 127.0.0.1:4567;
    server 127.0.0.1:4568;
    server 127.0.0.1:4569;
}

# the actual encrypted server
server {
  listen 443 ssl spdy;

	# change this!
	 server_name example.com;

  ssl on;
  
  # change this!
  ssl_certificate /etc/nginx/ssl/bundle.crt;
  
  #change this!
  ssl_certificate_key /etc/nginx/ssl/site.de.key;

  # enables all versions of TLS, but not SSLv2 or 3 which are weak and now deprecated.
  ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

  # disables all weak ciphers
	ssl_ciphers 'AES128+EECDH:AES128+EDH';
  ssl_prefer_server_ciphers on;

    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_set_header X-NginX-Proxy true;
    proxy_redirect off;

    # Socket.io Support
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    
    # watch out - no quotes!
    proxy_set_header Connection upgrade;

    gzip            on;
    gzip_min_length 1000;
    gzip_proxied    off;
    gzip_types      text/plain application/xml application/x-javascript text/css application/json;

    location @nodebb {
        proxy_pass http://io_nodes;
    }

    location ~ ^/(images|language|sounds|templates|uploads|vendor|src\/modules|nodebb\.min\.js|stylesheet\.css|admin\.css) {
       
        # change this! 
        root /var/www/nodebb/public/;
        try_files $uri $uri/ @nodebb;
    }

    location / {
			proxy_pass http://io_nodes;
    }

}

[nhl.pl]

An issue with terminating SSL connections?

Nginx, Websockets, SSL and Socket.IO deployment

[KingCat]

@Moritz-Friedrich Unfortunately none of those config changes made a difference I'm behind cloud flare ssl so I'm not sure if that's what's causing it (even though I disabled it completely for awhile and it still happened)

[Moritz Friedrich]

@KingCat , if you run sudo nginx -t, what happens exactly?

[KingCat]

@Moritz-Friedrich

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

[julian]

It's cloudflare. Keep using it, but disable acceleration. Websocket issues will be fixed.

[KingCat]

@julian said:

It's cloudflare. Keep using it, but disable acceleration. Websocket issues will be fixed.

I did what you suggested and it seems to have fixed the issue in chrome. However in safari it still happens. I'm gonna wait a while and see if maybe safari just has something weird cached (even though i cleared browser and dns cache).

[KingCat]

So it's been a few days and i'm still getting the errors in safari. The errors don't show up in any other browser though.

If I had to guess, I'd say theres some kind of technology or protocol that safari doesn't support (maybe the wss:// protocol?) so the socket.io has to fallback to using http instead and that seems to be why the error happens. Totally a guess though.

If this is the best I can do I'm OK with that as 0.1% of my users are on safari and it's not a site-breaking bug either. I'll consider this closed unless anyone has anything else to try.

Also, not sure if it matters but I'm using a free StartSSL certificate. It's not a self-signed one and it gives the green lock in safari but just in case it matters for some reason.

[KingCat]

By the way, I visited the link in the error messages in safari (https://forum.nzb.cat/socket.io/?EIO=3&transport=polling&t=1429031828694-10&sid=E7eEDbKPU53mKENvAAW3)

and got this:

{"code":1,"message":"Session ID unknown"}

Not sure what this means for the cause of the 400/502 errors.

[Schamper]

@KingCat I don't really have anything to contribute, just wanted to say that I saw your username and went "... NZBCat?" and lo and behold... Two things I use combined together

[KingCat]

@Schamper Haha yep. I may be the KingCat but even I need support sometimes

[KingCat]

Welp, I solved it.

Turns out I had the wrong directory specified in nginx for static assets. It was pointing to a different install of nodebb. For some reason that caused issues with socket.io (and only with safari/opera for some reason).

Now, here's the weird thing. I only recently started using nginx to serve the assets. Recently as in like this morning. Before that I simply had a proxy pass setup and let nodebb serve the assets. I'm not sure what was causing the 400/502 errors then but I'm just glad their solved now.