Debugging nodejs app.js at 100%
-
Post nodejs version disable all custom plugins and see if it still happens. Try updating to nodejs 4.5 and see if it happens.
We are not seeing this issue on this forum or the forums we host. So it's either a nodejs version issue or a custom plugin doing something that's locking the node process. Also check this out https://community.nodebb.org/topic/8998/nodebb-periodically-hangs and https://community.nodebb.org/topic/8789/infinite-loop
-
@baris said in Debugging nodejs app.js at 100%:
Post nodejs version disable all custom plugins and see if it still happens. Try updating to nodejs 4.5 and see if it happens.
We are not seeing this issue on this forum or the forums we host. So it's either a nodejs version issue or a custom plugin doing something that's locking the node process. Also check this out https://community.nodebb.org/topic/8998/nodebb-periodically-hangs and https://community.nodebb.org/topic/8789/infinite-loop
my node is 4.4.7, i tried to disable some plugins and see if it makes a difference
thansk for the linksright now stopping nodebb keeps the cores at 100% and i have to kill manually
root@w1-exodo:/var/foroexodo/nodebb# ./nodebb stop Stopping NodeBB. Goodbye! root@w1-exodo:/var/foroexodo/nodebb# htop root@w1-exodo:/var/foroexodo/nodebb# ps aux | grep nodebb root 4970 0.0 0.0 8100 948 pts/1 S+ 17:10 0:00 grep --color=auto nodebb root 18095 0.0 1.0 971188 65068 ? Ssl 09:27 0:00 /usr/bin/nodejs /var/foroexodo/nodebb/loader.js root@w1-exodo:/var/foroexodo/nodebb# kill -9 18095 root@w1-exodo:/var/foroexodo/nodebb# ps aux | grep nodebb root 5425 0.0 0.0 8100 948 pts/1 S+ 17:10 0:00 grep --color=auto nodebb root@w1-exodo:/var/foroexodo/nodebb# htop root@w1-exodo:/var/foroexodo/nodebb# ps aux | grep node root 5434 0.0 0.0 8100 948 pts/1 S+ 17:10 0:00 grep --color=auto node root 5791 0.0 0.0 52136 5444 ? Ss Aug29 0:55 /usr/sbin/munin-node root 18109 36.4 11.0 1861016 673192 ? Rl 09:27 168:42 /usr/bin/nodejs app.js root 18114 77.3 12.3 1917928 756612 ? Rl 09:27 358:27 /usr/bin/nodejs app.js root@w1-exodo:/var/foroexodo/nodebb# kill -9 18109 root@w1-exodo:/var/foroexodo/nodebb# ps aux | grep node root 5437 0.0 0.0 8100 948 pts/1 S+ 17:10 0:00 grep --color=auto node root 5791 0.0 0.0 52136 5444 ? Ss Aug29 0:55 /usr/sbin/munin-node root 18114 77.4 12.3 1917928 756612 ? Rl 09:27 358:40 /usr/bin/nodejs app.js root@w1-exodo:/var/foroexodo/nodebb# kill -9 18114 root@w1-exodo:/var/foroexodo/nodebb# ps aux | grep node root 5439 0.0 0.0 8100 944 pts/1 S+ 17:10 0:00 grep --color=auto node root 5791 0.0 0.0 52136 5444 ? Ss Aug29 0:55 /usr/sbin/munin-node -
@hukadan said in Debugging nodejs app.js at 100%:
I cannot help you with Node itself. But if I may, you should run NodeBB using an unprivileged user. You will save yourself a lot of trouble. May be you should also change the user running nginx from
nobodyto something more sensible likewwworwww-data.Also, the Node and NodeBB version would certainly help Node gurus.
Thanks for the tips
-
@baris said in Debugging nodejs app.js at 100%:
Post nodejs version disable all custom plugins and see if it still happens. Try updating to nodejs 4.5 and see if it happens.
We are not seeing this issue on this forum or the forums we host. So it's either a nodejs version issue or a custom plugin doing something that's locking the node process. Also check this out https://community.nodebb.org/topic/8998/nodebb-periodically-hangs and https://community.nodebb.org/topic/8789/infinite-loop
This happens at wtdwtf too - we have a script that kills an instance if it pegs a core and stops responding IIRC. I believe it's a nodejs bug.
ETA: The reason we suspect it's a nodejs bug, IIRC (it's been a while since we discussed it), is because it's happening in native code at low levels. I think it was suggested that specific virtualized environments are also related to the issue - DigitalOcean is the one we're using.
-
@sloosecannon said in Debugging nodejs app.js at 100%:
@baris said in Debugging nodejs app.js at 100%:
Post nodejs version disable all custom plugins and see if it still happens. Try updating to nodejs 4.5 and see if it happens.
We are not seeing this issue on this forum or the forums we host. So it's either a nodejs version issue or a custom plugin doing something that's locking the node process. Also check this out https://community.nodebb.org/topic/8998/nodebb-periodically-hangs and https://community.nodebb.org/topic/8789/infinite-loop
This happens at wtdwtf too - we have a script that kills an instance if it pegs a core and stops responding IIRC. I believe it's a nodejs bug.
ETA: The reason we suspect it's a nodejs bug, IIRC (it's been a while since we discussed it), is because it's happening in native code at low levels. I think it was suggested that specific virtualized environments are also related to the issue - DigitalOcean is the one we're using.
Weird
Thanls for the info
We should work all togetjer to try solve this
Can you share that script? -
-
I haven't seen problems like this either, running on node v4.3.2
-
It may be related to traffic and heavy usage though... this community isn't as heavily used as some other ones.
However that said, we do host some forums with heavy traffic and don't see issues like this, though they are spread out over multiple cores/servers so each individual NodeBB server handles a fraction of the load.
-
@julian said in Debugging nodejs app.js at 100%:
It may be related to traffic and heavy usage though... this community isn't as heavily used as some other ones.
However that said, we do host some forums with heavy traffic and don't see issues like this, though they are spread out over multiple cores/servers so each individual NodeBB server handles a fraction of the load.
Yes. You wont see this issue on low load communities probably, i dont call mine a heavy traffic anyways.
Our server is not phisical and it can handle much more traffic, 12 cores.
The problem must be some function that gets random core at 100% and stuck. That ends on some instances not working and nodebb stop doing nothing at all. -
@exodo said in Debugging nodejs app.js at 100%:
@sloosecannon said in Debugging nodejs app.js at 100%:
@baris said in Debugging nodejs app.js at 100%:
Post nodejs version disable all custom plugins and see if it still happens. Try updating to nodejs 4.5 and see if it happens.
We are not seeing this issue on this forum or the forums we host. So it's either a nodejs version issue or a custom plugin doing something that's locking the node process. Also check this out https://community.nodebb.org/topic/8998/nodebb-periodically-hangs and https://community.nodebb.org/topic/8789/infinite-loop
This happens at wtdwtf too - we have a script that kills an instance if it pegs a core and stops responding IIRC. I believe it's a nodejs bug.
ETA: The reason we suspect it's a nodejs bug, IIRC (it's been a while since we discussed it), is because it's happening in native code at low levels. I think it was suggested that specific virtualized environments are also related to the issue - DigitalOcean is the one we're using.
Weird
Thanls for the info
We should work all togetjer to try solve this
Can you share that script?I don't have access to it, but I'll mention the server admins and see if they can help.
@Ben-Lubar, @boomzilla - can either of you post the restart-script we use at WTDWTF?
-
-
@baris said in Debugging nodejs app.js at 100%:
@jarey We are using nodejs 4.5.0 here.
Just installed 4.5.0
Issue is still there -
@exodo I would imagine it's something like
node --prof app.js
