Again: «Looks like your connection to NodeBB was lost, please wait while we try to reconnect.» - in Docker Swarm

PitaJ

@mwaeckerlin there are certain options which are dependent on the order in which they're applied in the config. I'd recommend trying to cut down the config, at least that pertinent to the NodeBB forum, to the SSL config at the following link

NginX - NodeBB Documentation

Documentation portal for NodeBB Forum Software

(docs.nodebb.org)

mwaeckerlin

@Tom_Rade said in Again: «Looks like your connection to NodeBB was lost, please wait while we try to reconnect.» - in Docker Swarm:

The outside URL https://forum.mrw.sh is the one that should be inside config.json.

And the directory /etc/nodebb doesn't exist normally with the Docker image. The default, official docker image uses /usr/src/app.

As you see in my YAML, I create /etc/nodebb, as I explain above, this is because it is not possible to mount single file as volume when this file has to be created at first run. And as you also see, I had to replace the command.

mwaeckerlin

@PitaJ, what options depend on the order? I checked all lines in NginX - NodeBB Documentation and they all should be in my configuration too. I even enhanced my docker image so, that now line proxy_redirect off is possible (before that was the only difference). But with no result.

Is there any way to debug? It seems that your NodeBB generates a 403, so is there any possibility to debug why a 403 is generated, what options/parameters does NodeBB get?

Has anyone ever tried the combination to run NodeBB in a docker swarm environment behind a reverse-proxy?

mwaeckerlin

In the code I see nconf. This would help to override some configurations. But I didn't find any documentation on how to use nconf. How can I set configuration options using nconf?

I found this code in src/socket.io/index.js:

        if (process.env.NODE_ENV !== 'development') {
                var domain = nconf.get('cookieDomain');
                var parsedUrl = url.parse(nconf.get('url'));
                var override = nconf.get('socket.io:origins');
                if (!domain) {
                        domain = parsedUrl.hostname;    // cookies don't prov
ide isolation by port: http://stackoverflow.com/a/16328399/122353
                }

                if (!override) {
                        io.origins(parsedUrl.protocol + '//' + domain + ':*')
;
                        winston.info('[socket.io] Restricting access to origin: ' + parsedUrl.protocol + '//' + domain + ':*');
                 } else {
                        io.origins(override);
                        winston.info('[socket.io] Restricting access to origin: ' + override);
               }
   }

By now, I configured NODE_ENV=development, this solves my problem, but is only a workaround.

mwaeckerlin

See: https://socket.io/docs/server-api/

It is possible to pass a function instead of only an url to io.origins. This way, you can trace before you reject the access. That helps all of your users to debug the problem. I suggest that you change your code accordingly.

I'll fork and provide a patch…

mwaeckerlin

Ok, I traced down the problem:

I changed the socket origin configuration to:

		var originUrl = override ? override : parsedUrl.protocol + '//' + domain;
		winston.info('[socket.io] Restricting access to origin: ' + originUrl);
		io.origins((origin, callback) => {
			if (origin.startsWith(originUrl)) {
				return callback(null, true);
			} else {
				winston.error('[socket.io] rejecting origin: ' + origin);
				winston.error('[socket.io] expected origin: ' + originUrl);
				return callback('origin not allowed', false);
			}
		})

This is the error I get:

2018-11-24T10:40:41.056Z [66] - error: [socket.io] rejecting origin: *
2018-11-24T10:40:41.056Z [66] - error: [socket.io] expected origin: http://forum.mrw.sh

That means: The problem is not the configuration, the problem is detected origin!

I'll prepare a patch to enable NodeBB in a docker environment, including a Dockerfile that works.

mwaeckerlin

Ok, I fixed the problem and created a pull request. Please accept it as soon as possible.

This is then a working Docker Swarm compose file:

version: '3.3'
services:

  mongodb:
    image: mongo
    volumes:
      - type: bind
        source: /srv/volumes/forum-mrw-sh/mongodb
        target: /data/db

  nodebb:
    image: mwaeckerlin/nodebb
    ports:
      - 8036:4567
    volumes:
      - type: bind
        source: /srv/volumes/forum-mrw-sh/nodebb/config
        target: /usr/src/app/config
      - type: bind
        source: /srv/volumes/forum-mrw-sh/nodebb/uploads
        target: /usr/src/app/public/uploads

The image here is mwaeckerlin/nodebb, as soon as my fix has been pulled into your repository, that can be changed to nodebb/docker.

In the online-setup, chose MongoDB and thet the db-url to mongodb.

Tom Rade

@mwaeckerlin The way I solved it was to mount /usr/src/app/temp, then start the server, move all files to the temporary "temp" directory, then stop the server and change the mount point to /usr/src/app. That worked quite nicely.

I think all your plugins and some settings will get deleted once you restart the server if you only mount config and uploads.

mwaeckerlin

@Tom_Rade, mounting, stopping, moving, moving mountpoint is not the way how a proper docker installation should work.

Could you give me a list of all files that should remain persistent?

Yes, in my nextcloud docker image I had to persist the apps (=plugins) path too

PitaJ

@mwaeckerlin socket origin is restricted to prevent what is called "cross site websocket hijacking". This is when another site connects to the socket server and acts as if it's the NodeBB client.

The reason your detected origin is wrong is almost certainly a configuration issue, most likely in your reverse proxy setup. This is why I suggest trying a cut down nginx config, as nginx is responsible for passing down the origin header. proxy_redirect off is one of the order-dependent options.

But I didn't find any documentation on how to use nconf

nconf options are set within config.json, documented here:

config.json - NodeBB Documentation

Documentation portal for NodeBB Forum Software

(docs.nodebb.org)

Dev mode sets this to "*" which enables connections from anywhere.

Tom Rade

@mwaeckerlin I just did the entire /usr/src/app directory, not sure if that's correct - let me know if you think otherwise.

mwaeckerlin

@Tom_Rade, no, that's not correct. Docker (and clean programming) require a strict separation between code and data.

mwaeckerlin

@PitaJ, the problem is, that not the url setting is *, but the origin that arrives in socket.io.

With a reverse proxy, I can handle same origin in nginx, centralized before it passes through to NodeBB, even more: NodeBB always gets all requests from only one source, from the reverse proxy server. So I suggest that the same origin feature should be optional: On by default, but it should be possible to disable it in reverse proxy environments.

I see that the origin reported from socket.io is *, but I don't understand yet, why and how.

mwaeckerlin

@PitaJ, exactly which option in the suggested nginx configuration passes the origin to the backend NodeBB?

I want to understand. Up to now, everything sounds like voodoo, such as «certain options which are dependent on the order» or «almost certainly a configuration issue». So what exactly is the problem? Which options are responsible for setting the origin? Which options depend in the order? What is why the correct order? AFAIK, order is not relevant for all options. I would like to have an educated understanding of what happens and why, not something that works by coincidence.

PitaJ

@mwaeckerlin said in Again: «Looks like your connection to NodeBB was lost, please wait while we try to reconnect.» - in Docker Swarm:

@PitaJ, the problem is, that not the url setting is *, but the origin that arrives in socket.io.

Yes I understood your problem.

With a reverse proxy, I can handle same origin in nginx, centralized before it passes through to NodeBB, even more: NodeBB always gets all requests from only one source, from the reverse proxy server. So I suggest that the same origin feature should be optional: On by default, but it should be possible to disable it in reverse proxy environments.

Did you go to the link? You can configure the socket.io origins by adding the following to config.json:

"socket.io": {
  "origins": "*"
}

Them you can add the same origin check to nginx.

I see that the origin reported from socket.io is *, but I don't understand yet, why and how.

I'm no nginx wizard either. It's pretty much voodoo to me as well. That's why I suggested trying a cut down config. If a config based closely on the one in our docs worked, then it's likely possible to figure out what the issue is based on differences between them.

I don't have enough experience to diagnose exactly what is wrong.

exactly which option in the suggested nginx configuration passes the origin to the backend NodeBB?

I want to understand. Up to now, everything sounds like voodoo, such as «certain options which are dependent on the order» or «almost certainly a configuration issue». So what exactly is the problem? Which options are responsible for setting the origin? Which options depend in the order? What is why the correct order? AFAIK, order is not relevant for all options. I would like to have an educated understanding of what happens and why, not something that works by coincidence.

I don't know enough about nginx to answer these questions. I just know that our config does work, and yours doesn't.

One more thing: none of us core devs know much about docker, which is why the documentation and defaults aren't very good for it.

mwaeckerlin

Ok, I suggest two thing:

I'll help you to elaborate a Dockerfile that works fine and stable, also in swarm environments (next step will be OpenShift and upgrades).
It would be helpful to add logging information when socket.io rejects a connection, i.e. log the detected vs the configured origin.

Agree, @PitaJ, @Tom_Rade?