Very weird behaviour when hosting multiple instances

[Netto Hikari]

Hey, guys!

I'm currently facing very weird behaviour when hosting multiple instances of NodeBB. My setup is as follows, I'm running Arch Linux:

• 1 account to host NodeBB under
• 3 folders for 3 different NodeBB installations
• 3 different Redis databases
• 3 different ports used
• 3 different "secrets" used

Now, when I send a chat message to a friend, it will also appear on all other hosted forums. Same with notifications. What the heck happened here?

~Netto

[Alex]

The first thing that comes in my mind (I am sure that I'm wrong), is maybe due to sockets?

[baris]

Can you post your config.json files for each instance for reference.

[Netto Hikari]

@baris Sure thing!

Instance 1

{
"base_url": "https://hanasu.me",
"port": "2339",
"secret": "such_secret (I changed this)",
"bind_address": "0.0.0.0",
"database": "redis",
"secondary_database": "none",
"redis": {
    "host": "127.0.0.1",
    "port": "6379",
    "password": "very password",
    "database": "0"
},
"bcrypt_rounds": 12,
"upload_path": "/public/uploads",
"use_port": false,
"relative_path": ""
}

Instance 2

{
"base_url": "http://unser-schleswig.de",
"port": "2340",
"secret": "very_secret (changed again)",
"bind_address": "0.0.0.0",
"database": "redis",
"redis": {
    "host": "127.0.0.1",
    "port": "6379",
    "password": "wow",
    "database": "1"
},
"bcrypt_rounds": 12,
"upload_path": "/public/uploads",
"use_port": false,
"relative_path": ""
}

Instance 3

{
"base_url": "http://37.59.31.162",
"port": "4567",
"secret": "do I have to do that again?",
"bind_address": "0.0.0.0",
"database": "redis",
"redis": {
    "host": "127.0.0.1",
    "port": "6379",
    "password": "-",
    "database": "2"
},
"bcrypt_rounds": 12,
"upload_path": "/public/uploads",
"use_port": true,
"relative_path": ""
}

Never touched the config after installation. Just followed the docs and thats it.

[baris]

Your configs look good, when you get a chat notification on the other instances do you actually get the chat message? If you reload the page and open up your chats do you see chat messages from users on other forums? If not then this might be isolated to socket io and the redis store we use to pub/sub socket messages across worker processes.

[Netto Hikari]

Yes, the chat messages are actually delivered on other instances. After reload, the chat messages are not displayed any more.

This is @Master-Antonio 's forum with a chat between him and me. He has UID #1 and I got #3.

This is my forum where I got UID #1 and it seems as I'm chatting with myself.

[Master-Antonio]

@Netto-Hikari
Yes, i confirm.
This is big Problem.
And i confirm if reload the page message not appears.

[baris]

So the messages are not actually saved into the wrong database, its just socket io seems to deliver it to all the instances running on the same redis instance even though they use different database ids.

[Netto Hikari]

@baris That's correct.

[baris]

From redis docs

Database & Scoping
Pub/Sub has no relation to the key space. It was made to not interfere with it on any level, including database numbers.
Publishing on db 10, will be heard on by a subscriber on db 1.
If you need scoping of some kind, prefix the channels with the name of the environment (test, staging, production, ...).

https://redis.io/docs/interact/pubsub/

(redis.io)

So at this point nodebb wont work with 3 instances using the same redis instance on different databases.

You can however have 3 redis instances on different ports ie (6379, 6380, 6381 so on) and use db 0 for all of them.

[Netto Hikari]

@baris Okay, then I'll do that. Is there a way to easily dump and restore a single database from a redis instance?

[baris]

I think you will have to get a copy of your *.rdb file and copy it to all redis instances. I don't know a quick way of moving db 1 to db 0 etc.

[baris]

Also looks like this might be a non issue when we upgrade to socket io 1.0+ and use the redis-adapter seems like it supports prefixes https://github.com/Automattic/socket.io-redis/blob/master/index.js#L48

[Netto Hikari]

@baris Thank you so much for your help. I'm running 3 instances of redis now and it seems to be working.