hundreds of polling requests

TheBronx

I've been trying nginx configuration for hours but still can't make it work fine.
Some requests keep returning 400 Bad Request, with a message saying: "Session ID unknown"
Enabling the websocket log I've found that when this happens, the following message is printed in the log:
io: 0 emit [ 'disconnect', { reason: 'transport error', rooms: [ 'pLywexv6lX6jRFvcAAAA', 'online_guests' ] } ]

There are also some 502 Bad Gateway errors, but it all seems to be caused by Nginx not being correctly configured for NodeBB and long polling.

I'm not familiar with Nginx, which makes things harder. I've tried with a few configurations found here and there, like:

proxy_buffering off;
proxy_connect_timeout      60s;
proxy_send_timeout         60s;
proxy_read_timeout         60s;
keepalive_timeout          60s;
proxy_buffers 8 32k;
proxy_buffer_size 64k;
large_client_header_buffers 8 32k;

Some of them seem to increase 502 errors, decreasing bad requests, but that's all I can say, pretty useless.

What seems clear to me is:

CloudFlare is not the problem (at least not right now), cause I'm testing without it, directly targeting Nginx.
Socket.io alone (without NodeBB) doesn't seem to be the problem either, unless you remove ip_hash from upstreams.
There is something between nginx and nodebb that is making long polling request to "timeout"?
When I remove ip_hash from nginx upstreams in nodebb, then I start to see hundreds of requests, without sleeps in between. That's a good improvement!

What I'm noticing now is that whenever a request takes more than 10 seconds to respond, it fails with a bad request. But when the request gets answered before 10s, it works:
ten seconds.png

We are getting closer, I hope

<baris>

@TheBronx Hey that 10s leads me to believe it might be this piece of code.

NodeBB/src/webserver.js at master · NodeBB/NodeBB

Node.js based forum software built for the modern web - NodeBB/src/webserver.js at master · NodeBB/NodeBB

GitHub (github.com)

Try commenting it out or increasing it and see if it works.

TheBronx

@baris said:

@TheBronx Hey that 10s leads me to believe it might be this piece of code.

NodeBB/src/webserver.js at master · NodeBB/NodeBB

Node.js based forum software built for the modern web - NodeBB/src/webserver.js at master · NodeBB/NodeBB

GitHub (github.com)

Try commenting it out or increasing it and see if it works.

WHAT THE FUCK IS THAAAAAT haha, it works!!!!!!!!!!!!!!!!!!!!!!!!!
you guys are gonna pay for this xD

<baris>

Does it work with a value like 30? I'll increase it in core.

TheBronx

@baris said:

Does it work with a value like 30? I'll increase it in core.

it seems 30000 would be a good limit. anyways, that "magic number" should not be there hehe.

PD: thanks for the help guys! I would have spent a lot of hours configuring nginx @baris haha.

<baris>

@TheBronx If we don't specify it at all the default is 60seconds I believe.

TheBronx

https://nodejs.org/api/http.html#http_server_settimeout_msecs_callback
2 minutes, if that doc is not wrong

<baris>

Ahh correct, in any case I removed it from master so it will be the default now.

julian

@baris @TheBronx ...... how embarrassing

markkus

Updated to master & works like a charm ;).