NodeBB periodically hangs

[L33t]

Looks like moving to the latest node.js did not help. Just happened again.

[boomzilla]

We're running our instance with --perf-basic-prof-only-functions now and are able to generate flamegraphs. Just waiting for the servercooties to strike now...

Example flamegraph:

[jiangcaiyang]

Never heard of it...
I am a lab rat...

[boomzilla]

Here's a thing. Had a nodebb process pegging the CPU and ran the perf tool against it. It collected 2,967 identical stacks:

node 29869 cpu-clock: 
            3288d1761413 LazyCompile:~ /usr/src/app/node_modules/nodebb-plugin-mentions/library.js:165 (/tmp/perf-29869.map)
            3288d1772468 LazyCompile:~ /usr/src/app/node_modules/nodebb-plugin-mentions/library.js:239 (/tmp/perf-29869.map)
            3288d1770e25 LazyCompile:~ /usr/src/app/node_modules/nodebb-plugin-mentions/library.js:209 (/tmp/perf-29869.map)
            3288d135f1f9 LazyCompile:~ /usr/src/app/src/database/mongo/sorted.js:341 (/tmp/perf-29869.map)
            3288d179b3fa LazyCompile:* /usr/src/app/src/topics/unread.js:340 (/tmp/perf-29869.map)
            3288d15bd907 LazyCompile:*readableAddChunk _stream_readable.js:124 (/tmp/perf-29869.map)
            3288d13e12ce LazyCompile:*Readable.push _stream_readable.js:100 (/tmp/perf-29869.map)
            7f52c6fdbb45 init_cacheinfo (/lib/x86_64-linux-gnu/libc-2.19.so)

So now the mentions plugin is suspect #1. cc: @julianlam

[pichalite]

@boomzilla anything on the database resource usage stats at that point in time?

[baris]

@boomzilla maybe disable the plugin and let it run and see if you have the same problem. Do you guys get this hanging issue daily?

Could be related to https://github.com/NodeBB/NodeBB/issues/4395

[Ben Lubar]

@baris said in NodeBB periodically hangs:

Do you guys get this hanging issue daily?

We've been tracking the manual restarts here: https://what.thedailywtf.com/topic/19990/the-official-server-cooties-tracking-thread

[Ben Lubar]

Is anyone who is affected by this using the dbsearch plugin with a MongoDB database? I got rid of ours and the periodic hangs went away, at least for the last 24 hours.

[PitaJ]

@Ben-Lubar hmmm. If you have a lot of posts and dbsearch has a synchronous loop this could be the cause.

[AOKP]

To be honest I think that the actual issue here is a plugin or a poor setup overall, if not even a custom javascript interefering with one of NodeBB's internals. As of now I had no longs from 0.7.x and up.

How about adding additional info:

• NodeJS version
• Cluster?
• How do you start NodeBB?