Introducing the build system in v1.4.x

[julian]

It's been quite awhile since we've released a new version, and for the most part that is because we've been working hard to chase down some of the more esoteric bugs and building down technical debt, something that isn't all that sexy to discuss .

Some of the newer changes that have been brewing are the build system and asset routing system.

Click here to see the full blog post

[Jenkler]

Great stuff @julian!

A stable system is a happy one Bugs are important and yes boring but necessary to fix Nice blog post

[revir]

SO COOOOOOL!
Can't wait to try it.

[jiangcaiyang]

Asset system would be somewhat difficult to understand at the first glance, because we could not the details inside instead given the unused warning. I hope the warning and incompatibilities will be fixed somewhere in the future.

Moreover, is the "upload_path" in config.json bug fully fixed and applicable in newly version on NodeBB?

[julian]

@jiangcaiyang I believe it is not fixed for 1.4.3 but should be for the next version.

[teh_g]

@julian Will there be a guide for updating our nginx config or any other settings to leverage the new system?

[julian]

@teh_g Good idea, I will consider writing a short guide to changing the nginx config and verifying that it works

[baris]

https://docs.nodebb.org/en/latest/scaling/index.html#use-a-proxy-server-to-serve-static-assets

@julian this needs to be updated as well

[julian]

@baris Updated

[teh_g]

@julian said in Introducing the build system in v1.4.x:

@teh_g Good idea, I will consider writing a short guide to changing the nginx config and verifying that it works

Looks like there will be a few changes. In the past, the documentation had the proxy settings in the location block, now they are getting moved into server. So mine would go from this (redacted some information)

# Redirect all port 80 requests to https
server {
    listen 80;
    server_name gamingexodus.com www.gamingexodus.com;
    return 301 https://$host$request_uri;
}

# Real meat of the server block
server {
        listen 443 ssl http2;

<pulled out my SSL config>

        root /nodebb/folder;
        index index.html index.htm;

        # Make site accessible from http://localhost/
        server_name gamingexodus.com www.gamingexodus.com;

        location / {
                # First attempt to serve request as file, then
                # as directory, then fall back to displaying a 404.
                # try_files $uri $uri/ =404;
                # Uncomment to enable naxsi on this location
                # include /etc/nginx/naxsi.rules

                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header Host $http_host;
                proxy_set_header X-NginX-Proxy true;

                proxy_pass http://127.0.0.1:4567;
                proxy_redirect off;

                # Socket.IO Support
                proxy_http_version 1.1;
                proxy_set_header Upgrade $http_upgrade;
                proxy_set_header Connection "upgrade";
        }

        location /.well-known {
                root /redacted/letsencrypt/;
                allow all;
        }
}

To this?

# Redirect all port 80 requests to https
server {
    listen 80;
    server_name gamingexodus.com www.gamingexodus.com;
    return 301 https://$host$request_uri;
}

# Real meat of the server block
server {
        listen 443 ssl http2;

<pulled out my SSL config>

        # Hopefully resolve issues with image uploads
        client_max_body_size 20M;

	# Make site accessible from http://localhost/
        server_name gamingexodus.com www.gamingexodus.com;
		
	# Server block from NodeBB config page
		
	proxy_set_header X-Real-IP $remote_addr;
	proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
	proxy_set_header Host $http_host;
	proxy_set_header X-NginX-Proxy true;
	proxy_redirect off;
	
	# Socket.IO Support
	proxy_http_version 1.1;
	proxy_set_header Upgrade $http_upgrade;
	proxy_set_header Connection "upgrade";
		
	location @nodebb {
		proxy_pass http://127.0.0.1:4567;
	}

	location ~ ^/assets/(.*) {
		root /path/to/nodebb/;
		try_files /build/public/$1 /public/$1 @nodebb;
	}

	location /plugins/ {
		root /path/to/nodebb/build/public/;
		try_files $uri @nodebb;
	}

	location / {
		proxy_pass http://127.0.0.1:4567;
	}

        location /.well-known {
                root /redacted/letsencrypt/;
                allow all;
        }
}

It also sounds like I can make these changes as long as I am on 1.4.3?

[julian]

@teh_g Proxy settings can actually go anywhere. Specifically, proxy_set_header can belong in http, server, or location.

proxy_pass, of course, belongs in location, and doesn't really make sense in server.

[teh_g]

@julian said in Introducing the build system in v1.4.x:

@teh_g Proxy settings can actually go anywhere. Specifically, proxy_set_header can belong in http, server, or location.

proxy_pass, of course, belongs in location, and doesn't really make sense in server.

Perfect, I am definitely no an nginx guru, so I have no idea :P. I just based it off of what was in the new documentation. I can give it a whirl on my server if you think the cursory glance looks good

Only change I made from what was in the documentation is instead of specifying the io_nodes section, I just used the server 127.0.0.1:4567 since I do not have a cluster.

[julian]

@teh_g said in Introducing the build system in v1.4.x:

Only change I made from what was in the documentation is instead of specifying the io_nodes section, I just used the server 127.0.0.1:4567 since I do not have a cluster.

Yep, that should work fine as well.

[teh_g]

@julian said in Introducing the build system in v1.4.x:

@teh_g said in Introducing the build system in v1.4.x:

Only change I made from what was in the documentation is instead of specifying the io_nodes section, I just used the server 127.0.0.1:4567 since I do not have a cluster.

Yep, that should work fine as well.

I may swap it over to use two processes since my DigitalOcean host has two CPUs...

I will make the swap of the nginx config later today and let you know how it goes!

[PitaJ]

@teh_g the proxy settings need to go in the server block now because they must apply to the @nodebb virtual location as well as the / location, when you're using nginx to serve static assets.

[teh_g]

My website didn't blow up, so all must be well with the changes. Always scary making nginx changes like that...

Do I need to make any changes in /public or are there any files I should remove as duplicates?

[julian]

@PitaJ Are you sure? I would've thought the named block only acts like a variable, and the proxy headers apply to whatever location block they're called from.

[PitaJ]

@julian if the proxy headers are only in the / block, then the won't get applied when the fallback occurs if a file is missing, since they aren't in the @nodebb or /assets locations.

[BuZz]

This looks like it has actually made my configuration more complicated - as I run nodebb in a subfolder.

Here is my previous config - https://community.nodebb.org/topic/10187/serving-up-static-content-with-nginx-with-nodebb-running-from-subfolder/

the important bit being

    location /forum {
        alias /path/to/nodebb/public;
        try_files $uri @nodebb;
    }

so it first tries looks in public, then passes it onto nodebb - now it looks like I will need multiple blocks. Will you be doing any example configs for subfolders ? I realise a subfolder install may have been a mistake now (as there have been a lot of subfolder bugs in nodebb), but I don't really want to move everything.

Another question which I was wondering when originally doing my subfolder config - why was the old code

location ~ ^/(images|language|sounds|templates|uploads|vendor|src\/modules|nodebb\.min\.js|stylesheet\.css|admin\.css) {  
    root /path/to/nodebb/public/;
    try_files $uri $uri/ @nodebb;
}

rather than just something along the lines of

location / {  
    root /path/to/nodebb/public/;
    try_files $uri $uri/ @nodebb;
}

was it to avoid some file accesses ?

Thanks.

[BuZz]

Another note - is the config here correct - https://docs.nodebb.org/en/latest/scaling/#use-a-proxy-server-to-serve-static-assets

it references @nodebb; but there is no @nodebb - nor in the basic nginx config ? Unless I'm missing something ? [edit] see next reply