v1.6.0 Suspected Nginx Problem

Technical Support
  • Upgraded from v.1.5.3 to v.1.6.0 as per the upgrade manual.

    Browser fails to display pages when I follow a link. If the page is refreshed it loads but only displays the post text and not the markdown formatting.

    No erros from ./nodebb log

    Nginx error.log shows:

    2017/10/01 00:04:52 [error] 537#537: *184 connect() failed (111: Connection refused) while connecting to upstream, client: 86.30.244.235, server: myforum.com, request: "GET /socket.io/?EIO=3&transport=polling&t=LxL2GT8 HTTP/1.1", upstream: "http://127.0.0.1:4567/socket.io/?EIO=3&transport=polling&t=LxL2GT8", host: "myforum.com", referrer: "https://myforum.com/"

    My nginx configuration is from the docs with SSL.

    If I bypass Nginx and access port 4567 directly in the browser, the forum displays correctly. This leads me to believe the problem is todo with Nginx.

    Nodebb is running on Ubuntu 14.04, as per the installation instructions.

  • One possible reason may be, that your noddebb doesn't listen on localhost:4567, but on the local lan interface IP address only.

    You can check that with 'netstat -nlp | more'

  • Have you read my post Tips to get 1.6.0 running after updating from 1.5.x ?
    What you write is one of the problems I had. I solved adding two extra rules to nginx configuration.

  • Browser fails to display pages when I follow a link.

    Sounds like a Socket.IO misconfiguration.

    My nginx configuration is from the docs with SSL.

    I also had problems when I followed the docs. What I had to do was appending a definition the end of my config.json file:

    {
        "url": "https://mydomain.tld",
        "secret": "uuid4 string",
        [...]
        "socket.io": {
            "origins": "http://mydomain.tld:* https://mydomain.tld:*"
        }
    }
    

    That one tells to accept connections coming from that domain.

    If the problem persists, I also did some customizing my NGINX config:

    • /etc/nginx/sites-available/mydomain-tld-http.conf
    # redirect everything to HTTPS
    server {
           listen 80;
           listen [::]:80;
    
           server_name mydomain.tld *.mydomain.tld;
    
           include /etc/nginx/snippets/acme.conf;
    
           return 301 https://mydomain.tld$request_uri;
    }
    
    # remove all subdomains from HTTPS
    server {
           listen 443 ssl http2;
           listen [::]:443 ssl http2;
    
           server_name *.mydomain.tld;
    
           include /etc/nginx/snippets/tlsgzip.conf;
    
           return 301 https://mydomain.tld$request_uri;
    }
    
    # HTTPS forum
    server {
           listen 443 ssl http2;
           listen [::]:443 ssl http2;
    
           server_name mydomain.tld;
    
           root /var/www/html; #just a fallback
    
           location / {
                   proxy_set_header X-Real-IP $remote_addr;
                   proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                   proxy_set_header X-Forwarded-Proto $scheme;
                   proxy_set_header Host $http_host;
                   proxy_set_header X-NginX-Proxy true;
    
                   proxy_pass http://127.0.0.1:4567;
                   proxy_redirect off;
    
                   # Socket.IO Support
                   proxy_http_version 1.1;
                   proxy_set_header Upgrade $http_upgrade;
                   proxy_set_header Connection "upgrade";
           }
           include /etc/nginx/snippets/tlsgzip.conf;
    }
    

    As seen, there are some includes:

    • /etc/nginx/snippets/acme.conf
    location ^~ /.well-known/acme-challenge/ {
            default_type "text/plain";
            root         /var/www/html;
    }
    location = /.well-known/acme-challenge/ {
            return 404;
    }
    

    This is just a snippet that makes EFF's CertBot configuration simpler.

    • /etc/nginx/snippets/tlsgzip.conf
    ssl on;
    ssl_certificate /etc/letsencrypt/live/mydomain.tld/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/mydomain.tld/privkey.pem;
    
    ssl_protocols TLSv1.2 TLSv1.1;
    ssl_prefer_server_ciphers on;
    ssl_ciphers ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDH-ECDSA-AES256-GCM-SHA384:ECDH-ECDSA-AES256-SHA384:ECDH-RSA-AES256-GCM-SHA384:ECDH-RSA-AES256-SHA384:ECDH-ECDSA-AES128-GCM-SHA256:ECDH-ECDSA-AES128-SHA256:ECDH-RSA-AES128-GCM-SHA256:ECDH-RSA-AES128-SHA256:DHE-RSA-AES256-GCM-SHA384:!DHE-RSA-AES256-SHA256:!DHE-RSA-AES128-GCM-SHA256:!DHE-RSA-AES128-SHA256:!DES-CBC3-SHA:!aNULL:!eNULL:!ADH:!EXP:!LOW:!DES:!MD5:!PSK:!SRP:!DSS:!RC4:!DHE-RSA-AES256-GCM-SHA384:!DHE-RSA-CAMELLIA256-SHA:!DHE-RSA-AES128-SHA:!DHE-RSA-CAMELLIA128-SHA;
    
    ssl_session_cache shared:TLS:2m;
    
    ssl_stapling on;
    ssl_stapling_verify on;
    resolver 8.8.8.8 8.8.4.4 [2001:4860:4860::8888] [2001:4860:4860::8844];
    
    gzip on;
    gzip_comp_level 9;
    
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" always;
    

    This snippet just adds TLS certificates and enables GZIP at maximum compression.

    About the SSL cyphers, that's a huge list I copied from an old tutorial and updated myself in order to keep an A+ grade at SSL Labs.

    Nodebb is running on Ubuntu 14.04

    I'm running it on a Ubuntu 16.04 VPS.

  • @manolino thanks, yes before posting I did find your suggestion and tried it. Unfortunately it didn't work. I should have included that in the original post.

  • @sfner thanks for the detailed reply. I tried including socket.io definition to my config.json but it had no effect. I also tried adding all sub-domains to nginx conf but it too had no effect. For both efforts I checked the direct non-nginx and it worked.

    Perhaps I should have added that it's running on a VPS that uses openVZ.

  • @manolino here's the output:

    Active Internet connections (only servers)
    Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
    tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      566/sshd        
    tcp        0      0 0.0.0.0:4567            0.0.0.0:*               LISTEN      6572/node       
    tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN      531/nginx -g daemon
    tcp        0      0 0.0.0.0:17500           0.0.0.0:*               LISTEN      594/dropbox     
    tcp        0      0 127.0.0.1:17600         0.0.0.0:*               LISTEN      594/dropbox     
    tcp        0      0 127.0.0.1:17603         0.0.0.0:*               LISTEN      594/dropbox     
    tcp        0      0 127.0.0.1:27017         0.0.0.0:*               LISTEN      580/mongod      
    tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      531/nginx -g daemon
    tcp6       0      0 :::22                   :::*                    LISTEN      566/sshd        
    tcp6       0      0 :::443                  :::*                    LISTEN      531/nginx -g daemon
    tcp6       0      0 :::17500                :::*                    LISTEN      594/dropbox     
    tcp6       0      0 :::80                   :::*                    LISTEN      531/nginx -g daemon
    udp        0      0 0.0.0.0:17500           0.0.0.0:*                           594/dropbox     
    Active UNIX domain sockets (only servers)
    Proto RefCnt Flags       Type       State         I-Node   PID/Program name    Path
    unix  2      [ ACC ]     STREAM     LISTENING     2466870449 594/dropbox         /home/euan/.dropbox/command_socket
    unix  2      [ ACC ]     STREAM     LISTENING     2466870452 594/dropbox         /home/euan/.dropbox/iface_socket
    unix  2      [ ACC ]     STREAM     LISTENING     2466863845 580/mongod          /tmp/mongodb-27017.sock
    unix  2      [ ACC ]     STREAM     LISTENING     2466865203 1121/python         /var/run/fail2ban/fail2ban.sock
    unix  2      [ ACC ]     STREAM     LISTENING     2466861640 1/init              @/com/ubuntu/upstart
    unix  2      [ ACC ]     SEQPACKET  LISTENING     2466862203 156/systemd-udevd   /run/udev/control
    unix  2      [ ACC ]     STREAM     LISTENING     2466863783 578/php-fpm.conf)   /run/php/php5.6-fpm.sock
    unix  2      [ ACC ]     STREAM     LISTENING     2466864056 663/dirmngr         /var/run/dirmngr/socket
    
  • As a last random shot, the tutorial instructs you installing the 6.x version of NodeJS, but I ignored that and installed the 8.x one.

    curl -sL https://deb.nodesource.com/setup_8.x | sudo -E bash -
    sudo apt-get install -y nodejs
    

    Installing Node.js via package manager

    (I did the same thing for MongoDB (installing the latest version, ignoring the installing instructions), but as the database is working for you, I wouldn't touch it for now)

  • @unterm said in v1.6.0 Suspected Nginx Problem:

    If I bypass Nginx and access port 4567 directly in the browser, the forum displays correctly. This leads me to believe the problem is todo with Nginx.

    I simply skipped this. Sorry.

    Try adding the NGINX's stable PPA and then upgrading the packages from your system:

  • @sfner Make sure [email protected] is installed, not v5.

  • Hi everyone,
    I'm having the same issue... I reviewed and tried all the links here, also, if I bypass to port 4567 my forum works... I just ran out of options... do you have any updates? (I do not want to open another thread for the same issue)

    Cheers
    Christian

  • @christian-mendieta You'll probably want to share your nginx config for NodeBB

  • @julian sure:

    #This is a redirect to allow only secure connections
    server {
    	listen          80;
    	server_name     mysite.tld;
    	return 302 https://$server_name$request_uri;
    }
    server {
    	listen 443 ssl;
    	listen [::]:443 ssl;
    	include snippets/ssl-mysite.tld.conf;
    	include snippets/ssl-params.conf;
    	server_name mysite.tld;
    
    	root /var/www/html/mysite.tld/public_html;
    	index index.php index.html;
    
    	access_log /var/log/nginx/mysite.tld.access.log;
    	error_log /var/log/nginx/mysite.tld.error.log;
    
    	# SSL block
    	location ~ /.well-known {
    			allow all;
    	}
    
    	# Deny access to .htaccess
    	location ~ /\.ht {
    			deny all;
    	}
    
    	location / {
    		proxy_set_header X-Real-IP $remote_addr;
    		proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    		proxy_set_header X-Forwarded-Proto $scheme;
    		proxy_set_header Host $host;
    		proxy_set_header X-NginX-Proxy true;
    
    		proxy_pass http://127.0.0.1:4567;
    		proxy_redirect off;
    
    		# New fixes. Values are powers of 2, this works for me, you can increase.
    		#proxy_headers_hash_bucket_size  128;
    		#proxy_headers_hash_max_size  1024;
    
    		# Socket.IO Support
    		proxy_http_version 1.1;
    		proxy_set_header Upgrade $http_upgrade;
    		proxy_set_header Connection "upgrade";
    	} 
    }
    

    Am I missing something?
    Thanks!
    Christian

  • What's your config.json?

  • That would be:

    {
        "url": "https://mysite.tld",
        "secret": "97a8ebc8-dxx",
        "database": "mongo",
        "port": 4567,
        "mongo": {
            "host": "192.168.xx.xx",
            "port": "27097",
            "username": "nodebb",
            "password": "soeasy",
            "database": "nodebb"
        }
    }
    
  • @Christian-Mendieta

    Any chance you can reset your config to just the bare-bones as defined in https://docs.nodebb.org/configuring/proxies/nginx/?

    I'm looking at the includes, which could literally be anything 😕

  • OK, did some clean up, the problem persist, the nginx config now is:

    server {
            listen          80;
            server_name     mysite.tld;
            return 302 https://$server_name$request_uri;
    }
    
    #This is a redirect to allow only secure connections
    server {
    	listen 443 ssl;
    	listen [::]:443 ssl;
    
    	ssl_certificate /etc/letsencrypt/live/mysite.tld/fullchain.pem;
    	ssl_certificate_key /etc/letsencrypt/live/mysite.tld/privkey.pem;
    
    	ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    	ssl_prefer_server_ciphers on;
    	ssl_ciphers 'AES128+EECDH:AES128+EDH';
    
    	ssl_dhparam /etc/ssl/certs/dhparam.pem;
    	server_name mysite.tld;
    
    	root /var/www/html/mysite.tld/public_html;
    	index index.php index.html;
    
    	access_log /var/log/nginx/mysite.tld.access.log;
    	error_log /var/log/nginx/mysite.tld.error.log;
    
    	# SSL block
    	location ~ /.well-known {
    			allow all;
    	}
    
    	# Deny access to .htaccess
    	location ~ /\.ht {
    			deny all;
    	}
    
    	location / {
    		 proxy_set_header X-Real-IP $remote_addr;
    		 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    		 proxy_set_header X-Forwarded-Proto $scheme;
    		 proxy_set_header Host $host;
    		 proxy_set_header X-NginX-Proxy true;
    
    		 proxy_pass http://127.0.0.1:4567;
    		 proxy_redirect off;     
    		 # New fixes. Values are powers of 2, this works for me, you can increase.
    		 #proxy_headers_hash_bucket_size  128;
    		 #proxy_headers_hash_max_size  1024;
    
    		 # Socket.IO Support
    		 proxy_http_version 1.1;
    		 proxy_set_header Upgrade $http_upgrade;
    		 proxy_set_header Connection "upgrade";
    	}
    
    }
    
  • Try removing

    root /var/www/html/mysite.tld/public_html;
    index index.php index.html;
    

    Probably won't fix it but it's worth a try. Also, validate your nginx configs and try restarting nginx completely (as opposed to reloading)

  • Thanks for your answer, and you're right it didn't fix the problem even with nginx restart.

  • Hi,
    Just for the record, I managed to make it work, here's the nginx config:

    server {
        listen      80;
        server_name mydomain.tls;
        return 301  https://$server_name$request_uri;
    }
    
    
    server {
        server_name mydomain.tls;
    
        listen 443 ssl; # managed by Certbot
        ssl_certificate /etc/letsencrypt/live/mydomain.tls/fullchain.pem; # managed by Certbot
        ssl_certificate_key /etc/letsencrypt/live/mydomain.tls/privkey.pem; # managed by Certbot
        include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
        ssl_dhparam /etc/ssl/certs/dhparam.pem;
    
        location / {
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
            proxy_set_header Host $http_host;
            proxy_set_header X-NginX-Proxy true;
    
            proxy_pass http://127.0.0.1:4567;
            proxy_redirect off;
    
            # Socket.IO Support
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
        }
    
    }
    

    Hope it helps someone.
    Cheers
    Christian


Suggested Topics


| | | |