socket.io log flooding

[PitaJ]

are you using cloudflare?

looks like your sockets aren't working right. Can you show client-side logs too?

[xidui]

@PitaJ
Thanks for the hint.
The client side log is described here
https://community.nodebb.org/topic/9752/some-polling-requests-fail

[xidui]

@PitaJ
We run it on linode machine, not cloudflare.

By the way, we had run it on Digital Ocean machine, it also behave like this.

[PitaJ]

Yeah it looks like it's failing to connect directly to websockets and is failing back on long polling.

[xidui]

@PitaJ

Can we disable socket?

[PitaJ]

@xidui no you need sockets for NodeBB to work right.

[xidui]

I notice that when the site is at peak hour, most of the logs are io logs such as disconnecting or disconnect.
When not during the peak hours, such error becomes less but still exists.

Seems the nginx config is right, other wise socket can not work even during valley hours.

@administrators
I wonder is their any limit of user in NodeBB. During peak hour, users active exceeds 1000 monitors at admin page, is that size too large for Nodebb to hold in on single machine?

[xidui]

I added this config:

"socket.io": {
    "transports": ["websocket"]
}

The flooding error seems less than before. But still need to monitor over several days.

[baris]

If you have more than 1000 users online on a single process you might be hitting open file limits on your OS try increasing those. https://easyengine.io/tutorials/linux/increase-open-files-limit/

[xidui]

@baris

That's single machine, 4 processes and the 1000 is active user not online user.

[xidui]

@xidui said in socket.io log flooding:

I added this config:

"socket.io": {
    "transports": ["websocket"]
}

The flooding error seems less than before. But still need to monitor over several days.

The performance seems better after I applied this change in config.

1. Behaviors before change:

   • Most of the logs are io emit timeout or io emit transport close . Only very rare portion (less than 10 percent) of requests successfully processed.
   • Every time I restart the site it can not response for about 5 minutes or longer even I see in the log that process has already listened to 4567.
   • Some of the socket.io/EIO?xxxx requests failed at client side.

2. Behaviors after change:

   • No io emit timeout was found ever
   • io emit transport close still exists, about 50% or less at peak hour and 10% at valley hour.
   • The site is fast enough at peak hour and it can recover quickly even I restart.
   • None of the requests failed at client side.

@administrators
I think my issue has accidentally and surprisingly solved by this small change temporarily. I wonder why and is this config recommended or ["polling", "websocket"] is better? And do you have some idea on the root cause of the issue when I use ["polling", "websocket"]?

[xidui]

Thanks for every one!

This issue seems to disappear after the change in config. Our site is more stable than ever before in the last week with a rather short response time during the whole week.

Thanks again and welcome to have a visit. Advices on the site is always welcomed.