@exodo said in Debugging nodejs app.js at 100%:

@sloosecannon said in Debugging nodejs app.js at 100%:

@baris said in Debugging nodejs app.js at 100%:

Post nodejs version disable all custom plugins and see if it still happens. Try updating to nodejs 4.5 and see if it happens.

We are not seeing this issue on this forum or the forums we host. So it's either a nodejs version issue or a custom plugin doing something that's locking the node process. Also check this out https://community.nodebb.org/topic/8998/nodebb-periodically-hangs and https://community.nodebb.org/topic/8789/infinite-loop

This happens at wtdwtf too - we have a script that kills an instance if it pegs a core and stops responding IIRC. I believe it's a nodejs bug.

ETA: The reason we suspect it's a nodejs bug, IIRC (it's been a while since we discussed it), is because it's happening in native code at low levels. I think it was suggested that specific virtualized environments are also related to the issue - DigitalOcean is the one we're using.

Weird
Thanls for the info
We should work all togetjer to try solve this
Can you share that script?

I don't have access to it, but I'll mention the server admins and see if they can help.

@Ben-Lubar, @boomzilla - can either of you post the restart-script we use at WTDWTF?

@baris said in Debugging nodejs app.js at 100%:

Post nodejs version disable all custom plugins and see if it still happens. Try updating to nodejs 4.5 and see if it happens.

We are not seeing this issue on this forum or the forums we host. So it's either a nodejs version issue or a custom plugin doing something that's locking the node process. Also check this out https://community.nodebb.org/topic/8998/nodebb-periodically-hangs and https://community.nodebb.org/topic/8789/infinite-loop

This happens at wtdwtf too - we have a script that kills an instance if it pegs a core and stops responding IIRC. I believe it's a nodejs bug.

ETA: The reason we suspect it's a nodejs bug, IIRC (it's been a while since we discussed it), is because it's happening in native code at low levels. I think it was suggested that specific virtualized environments are also related to the issue - DigitalOcean is the one we're using.

Hmm, so I just found an interesting behavior that might be the cause.

It appears that NodeBB uses the websocket by default, but when the connection fails, it falls back to polling!

It does not appear to reconnect to the websocket after the failure. There's an easy way to see this - open up Chrome's dev tools on a NodeBB site, then kill your network connection.
It starts sending polling requests, and continues even after the connection is re-established!

Perhaps this is the reason that it gets "stuck" and has weird connectivity bugs?

@teh_g Yep, I got that as well. That looks much better, but it still was stuck even when the connection was recovered (The submit button was disabled)

@baris That would be nice. Also, probably not a bad idea to add some kind of UI cue to indicate that the button has been pressed, and maybe something in the way of error-checking to let people know when the socket is broken (That would be more of a general thing - there are a lot of actions that don't work at all without any kind of feedback - upvoting, scrolling to load posts, IIRC some of the buttons too).

@julian said in RFC: Make the forum clients more resilient to bad connections:

Hey @sloosecannon -- good topic

In general, the times when connection drops are most noticible are during:

1. Page navigations
2. Posts

Other real-time things you honestly can't tell, since if someone posts and it doesn't show up right away, it's not really all that detrimental to the end user experience

Now, the former, we really can't do anything about, since if the call to the api route takes awhile, we just have to wait... the latter, we could. A post could be rendered on the client-side, and then stored in the active session while it goes about saving to the backend.

The risk, of course, is that the user might leave the site, at which point poof, goes the transient data.

Not so good

Right - I believe Discourse does something like that as well. And it does indeed have exactly that issue

I'd be interested in a way to debug what exactly has broken down in socket.io when these drops happen - it seems like it sometimes just refuses to reconnect without a page refresh, and I think that's the thing people find most annoying - it's not necessarily the fact that it breaks, it's that it breaks and stays broken!

@torn2 said in RFC: Make the forum clients more resilient to bad connections:

Man, we luckily have not had any issues like this. Makes you not want to upgrade and/or make changes. Which version are you guys using???

Bleeding-edge-ish latest, updated from Git. Not sure what we're on right now, don't feel like looking up the thread

But we've had these issues pretty much since we started using it.

@exodo said in RFC: Make the forum clients more resilient to bad connections:

Agree. It happens a lot on 3g navigation

Yeah. We have an ongoing issue of NodeBB pegging a CPU core too, and it happens a lot when it does that (we need to restart the nodejs process).

So I'm not sure where exactly I should put this, but I figured "feature requests" is probably most appropriate (as it's not a single "bug", exactly...)

So, the NodeBB forum I am with is full of picky software developers who complain about the forum all the time

One of the biggest, if not the biggest complaint, is how badly NodeBB drops connection on bad internet connections, namely mobile or WiFi hotspot hopping.

I'd like to post this thread as more of a discussion of how to fix it - I'm not exactly familiar with the codebase, and I don't really know how the best way to handle it would be, but it seems like socket.io, or the implementation NodeBB has of it, breaks badly and often in unreliable conditions (leading to pages failing to load, votes failing to take effect, and the reply/quote buttons failing to have any effect, usually requiring a refresh of the page).

I'll probably be taking a closer look at the client side code, but I'm curious if anyone else has any ideas of how to fix this particular issue - or whether due to socket.io problems, it can be fixed at all.