Introducing the build system in v1.4.x

[julian]

@teh_g said in Introducing the build system in v1.4.x:

Only change I made from what was in the documentation is instead of specifying the io_nodes section, I just used the server 127.0.0.1:4567 since I do not have a cluster.

Yep, that should work fine as well.

[teh_g]

@julian said in Introducing the build system in v1.4.x:

@teh_g said in Introducing the build system in v1.4.x:

Only change I made from what was in the documentation is instead of specifying the io_nodes section, I just used the server 127.0.0.1:4567 since I do not have a cluster.

Yep, that should work fine as well.

I may swap it over to use two processes since my DigitalOcean host has two CPUs...

I will make the swap of the nginx config later today and let you know how it goes!

[PitaJ]

@teh_g the proxy settings need to go in the server block now because they must apply to the @nodebb virtual location as well as the / location, when you're using nginx to serve static assets.

[teh_g]

My website didn't blow up, so all must be well with the changes. Always scary making nginx changes like that...

Do I need to make any changes in /public or are there any files I should remove as duplicates?

[julian]

@PitaJ Are you sure? I would've thought the named block only acts like a variable, and the proxy headers apply to whatever location block they're called from.

[PitaJ]

@julian if the proxy headers are only in the / block, then the won't get applied when the fallback occurs if a file is missing, since they aren't in the @nodebb or /assets locations.

[BuZz]

This looks like it has actually made my configuration more complicated - as I run nodebb in a subfolder.

Here is my previous config - https://community.nodebb.org/topic/10187/serving-up-static-content-with-nginx-with-nodebb-running-from-subfolder/

the important bit being

    location /forum {
        alias /path/to/nodebb/public;
        try_files $uri @nodebb;
    }

so it first tries looks in public, then passes it onto nodebb - now it looks like I will need multiple blocks. Will you be doing any example configs for subfolders ? I realise a subfolder install may have been a mistake now (as there have been a lot of subfolder bugs in nodebb), but I don't really want to move everything.

Another question which I was wondering when originally doing my subfolder config - why was the old code

location ~ ^/(images|language|sounds|templates|uploads|vendor|src\/modules|nodebb\.min\.js|stylesheet\.css|admin\.css) {  
    root /path/to/nodebb/public/;
    try_files $uri $uri/ @nodebb;
}

rather than just something along the lines of

location / {  
    root /path/to/nodebb/public/;
    try_files $uri $uri/ @nodebb;
}

was it to avoid some file accesses ?

Thanks.

[BuZz]

Another note - is the config here correct - https://docs.nodebb.org/en/latest/scaling/#use-a-proxy-server-to-serve-static-assets

it references @nodebb; but there is no @nodebb - nor in the basic nginx config ? Unless I'm missing something ? [edit] see next reply

[BuZz]

@BuZz

sorry I misread / missed the top @nodebb - but it doesn't look right with

location @nodebb {
    proxy_pass http://127.0.0.1:4567;
}

location ~ ^/assets/(.*) {
    root /path/to/nodebb/;
    try_files /build/public/$1 /public/$1 @nodebb;
}

location /plugins/ {
    root /path/to/nodebb/build/public/;
    try_files $uri @nodebb;
}

location / {
    proxy_pass http://io_nodes;
}

location / gets passed onto io_nodes which contains the upstream servers, but location @nodebb gets passed onto a specific nodebb only ? I guess that should contain proxy_pass http://io_nodes; also ?

I realise this is additional config from the basic - but then the original config already has a location / in it with the proxy settings, which won't be used for the @nodebb location.

Would be interested in what a "subfolder" config should look like, as the one I have just been making seems a little over complicated - it would be nice to just have a single location with try_files as I had before, which checks public folder / build etc, then passes it onto nodebb if nothing found.

[edit] I see the proxy settings location was mentioned previously - but the documentation isn't updated to move the proxy settings inside the server block, which would be needed with the new config I think.

I also have now noticed the "full config" after which looks correct and has both locations referencing http://io_nodes; and has the proxy settings under the server.

[PitaJ]

@BuZz I don't think your previous config would have worked to speed up file requests at all. Odds are if you look at the response headers for static resources, the X-Powered-By: NodeBB header would be there (which indicates the files were served by NodeBB, not nginx). This is due to alias not working as expected with try_files.

was it to avoid some file accesses ?

Yes. Attempting to access a file before every request is unnecessary latency.

location / gets passed onto io_nodes which contains the upstream servers, but location @nodebb gets passed onto a specific nodebb only ? I guess that should contain proxy_pass http://io_nodes; also ?

Yes, this is a typo.

Here's an example config for a subfolder:

upstream io_nodes {
    ip_hash;
    server 127.0.0.1:4567;
    server 127.0.0.1:4568;
    server 127.0.0.1:4569;
}

server {
    listen 80;

    server_name community.nodebb.org;

    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_set_header X-NginX-Proxy true;
    proxy_redirect off;

    # Socket.io Support
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";

    gzip            on;
    gzip_min_length 1000;
    gzip_proxied    off;
    gzip_types      text/plain application/xml text/javascript application/javascript application/x-javascript text/css application/json;

    location @nodebb {
        proxy_pass http://io_nodes;
    }

    location ~ ^/forum/assets/(.*) {
        root /path/to/nodebb/;
        try_files /build/public/$1 /public/$1 @nodebb;
    }

    location ~ ^/forum/plugins/(.*) {
        root /path/to/nodebb/build/public/;
        try_files /plugins/$1 @nodebb;
    }

    location /forum/ {
        proxy_pass http://io_nodes;
    }
}

it would be nice to just have a single location with try_files as I had before, which checks public folder / build etc, then passes it onto nodebb if nothing found.

I wouldn't recommend a setup like you describe due to the previously mentioned extra latency. It also is likely impossible with our new double-directory approach.

EDIT: fixed a couple things

move the proxy settings inside the server block, which would be needed with the new config I think.

This is also true. In fact, I think the original config should just not have the proxy settings inside the location block in the first place.

[BuZz]

@PitaJ Thanks for the clarifications.

Regarding latency - some time ago I read about avoiding regular expression matching in nginx for performance - I am wondering if there is any improvements could be made to avoid the assets RE - I guess not due to the replace, but isn't that try_files going to cause some file accesses anyway ?

for the plugins I guess something like this could be used ?

  location /forum/plugins {
    alias /path/to/nodebb/build/public/plugins;
    try_files $uri @nodebb;
  }

? (untested, and written quickly though so I may have missed something).

[PitaJ]

@BuZz

some time ago I read about avoiding regular expression matching in nginx for performance

This is true, and I tried to avoid these RegExps, but it simply isn't possible until alias + try_files is fixed.

isn't that try_files going to cause some file accesses anyway

Yes, but unlike your previous solution which would attempt a file access on every single request to the site, this only attempts a file access when a request is made to the assets route.

for the plugins I guess something like this could be used ?

Like I said earlier, using alias with try_files does not work as expected in nginx. This particular case may work since there's only a single try_files path to try, but I haven't tested it, and can only be sure that what I provided in the example will work.

[BuZz]

@PitaJ I wasn't aware of the bug - reading about it now - 5 years old