Nodebb v1.11.1 suddenly started crashing

[fais3000]

Our forum had been running on this update for about a week. Today, it suddenly started crashing at following error.

2019-01-19T04:43:05.818Z [4567/22239] - verbose: [user/jobs] (Re-)starting user jobs...
2019-01-19T04:43:05.820Z [4567/22239] - verbose: [user/jobs] Starting job (digest.daily)
2019-01-19T04:43:05.824Z [4567/22239] - verbose: [user/jobs] Starting job (digest.weekly)
2019-01-19T04:43:05.826Z [4567/22239] - verbose: [user/jobs] Starting job (digest.monthly)
2019-01-19T04:43:05.827Z [4567/22239] - verbose: [user/jobs] Starting job (reset.clean)
2019-01-19T04:43:05.827Z [4567/22239] - verbose: [user/jobs] 4 jobs started
2019-01-19T04:43:05.828Z [4567/22239] - verbose: [emailer] Setting up SMTP fallback transport
2019-01-19T04:43:05.893Z [4567/22239] - verbose: [middleware/autoLocale] Retrieves languages list for middleware
2019-01-19T04:43:05.930Z [4567/22239] - info: [themes] Using theme nodebb-theme-persona
2019-01-19T04:43:05.931Z [4567/22239] - verbose: [plugins] Initializing plugins system
2019-01-19T04:43:05.992Z [4567/22239] - verbose: [emailer] Built custom email templates
2019-01-19T04:43:06.000Z [4567/22239] - verbose: [plugins] Loaded plugin: nodebb-theme-persona
2019-01-19T04:43:06.034Z [4567/22239] - verbose: [plugins] Loaded plugin: nodebb-plugin-dbsearch
2019-01-19T04:43:06.063Z [4567/22239] - verbose: [plugins] Loaded plugin: nodebb-widget-essentials
2019-01-19T04:43:06.069Z [4567/22239] - error: uncaughtException: No more documents in tailed cursor
MongoError: No more documents in tailed cursor
    at Function.MongoError.create (/var/www/community/node_modules/mongodb-core/lib/error.js:31:11)
    at nextFunction (/var/www/community/node_modules/mongodb-core/lib/cursor.js:653:50)
    at /var/www/community/node_modules/mongodb-core/lib/cursor.js:602:7
    at queryCallback (/var/www/community/node_modules/mongodb-core/lib/cursor.js:232:18)
    at /var/www/community/node_modules/mongodb-core/lib/connection/pool.js:469:18
    at _combinedTickCallback (internal/process/next_tick.js:131:7)
    at process._tickCallback (internal/process/next_tick.js:180:9) {"error":{"name":"MongoError","message":"No more documents in tailed cursor","tailable":true,"awaitData":true},"stack":"MongoError: No more documents in tailed cursor\n    at Function.MongoError.create (/var/www/community/node_modules/mongodb-core/lib/error.js:31:11)\n    at nextFunction (/var/www/community/node_modules/mongodb-core/lib/cursor.js:653:50)\n    at /var/www/community/node_modules/mongodb-core/lib/cursor.js:602:7\n    at queryCallback (/var/www/community/node_modules/mongodb-core/lib/cursor.js:232:18)\n    at /var/www/community/node_modules/mongodb-core/lib/connection/pool.js:469:18\n    at _combinedTickCallback (internal/process/next_tick.js:131:7)\n    at process._tickCallback (internal/process/next_tick.js:180:9)","exception":true,"date":"Sat Jan 19 2019 04:43:06 GMT+0000 (UTC)","process":{"pid":22239,"uid":0,"gid":0,"cwd":"/var/www/community","execPath":"/root/.nvm/versions/node/v8.9.3/bin/node","version":"v8.9.3","argv":["/root/.nvm/versions/node/v8.9.3/bin/node","/var/www/community/app.js"],"memoryUsage":{"rss":163893248,"heapTotal":120909824,"heapUsed":86452080,"external":45515363}},"os":{"loadavg":[4.00830078125,3.794921875,3.6513671875],"uptime":4430007},"trace":[{"column":11,"file":"/var/www/community/node_modules/mongodb-core/lib/error.js","function":"Function.MongoError.create","line":31,"method":"create","native":false},{"column":50,"file":"/var/www/community/node_modules/mongodb-core/lib/cursor.js","function":"nextFunction","line":653,"method":null,"native":false},{"column":7,"file":"/var/www/community/node_modules/mongodb-core/lib/cursor.js","function":null,"line":602,"method":null,"native":false},{"column":18,"file":"/var/www/community/node_modules/mongodb-core/lib/cursor.js","function":"queryCallback","line":232,"method":null,"native":false},{"column":18,"file":"/var/www/community/node_modules/mongodb-core/lib/connection/pool.js","function":null,"line":469,"method":null,"native":false},{"column":7,"file":"internal/process/next_tick.js","function":"_combinedTickCallback","line":131,"method":null,"native":false},{"column":9,"file":"internal/process/next_tick.js","function":"process._tickCallback","line":180,"method":"_tickCallback","native":false}]}
2019-01-19T04:43:06.070Z [4567/22239] - error: No more documents in tailed cursor {"name":"MongoError","tailable":true,"awaitData":true}
2019-01-19T04:43:06.070Z [4567/22239] - info: [app] Shutdown (SIGTERM/SIGINT) Initialised.
[cluster] Child Process (22239) has exited (code: 1, signal: null)
[cluster] Spinning up another process...

Nodebb v1.11.1
Mongodb version 3.2 and no replica's

• Tried to move to master branch but to no avail.
• Tried a new installation current database, no success

It seems like there have been some change to database that it fails.

Related Stack overflow post https://stackoverflow.com/questions/36704040/mongoerror-no-more-documents-in-tailed-cursor

@PitaJ @barisusakli Out of ideas, please help.

[faizanzahid]

having same issues, @julian do you guys have any tech support team who can help us? @baris

[baris]

Can you try installing redis and setting it up as your session store. See if the error goes away.

[fais3000]

@baris Ok, not sure how to add that in addition to an existing mongo db connection. I'll read and try out.

It seems that something changed in v1.11.1 which makes it not completely compatible with mongodb 3.2, which made our database unable to work anymore. Tried the same database with a new installation on 3.4. That error goes away but nodebb start script becomes too slow (takes many 10s of sec on each line) and web server doesn't serve the frontend.

[jtsimoes]

I'm also noticing that Nodebb is much slower, especially when starting and within the ACP.

And I noticed that sometimes when the forum stays a long time without anyone accessing, when I access through the link/browser, it displays the page with the error 503 - Excessive Load Warning. Idk why...

This could be some problem with my Ubuntu server (although I have already restarted it and updated it), could be some outdated plugins no longer compatible with NodeBB latest version too or even perfomance problems with NodeBB itself.

I don't really know. It's still very functional anyways

[faizanzahid]

@fais3000 any updates on this that u tried?

[fais3000]

@faizanzahid Reverted back to Nodebb 1.10.2 and moved mongo db to 3.4. Unfortunately database couldn't be resurrected, we had to use a backup dump.

[scottalanmiller]

1.11.2 is out now as well, might be worth updating before going to deep in any additional testing.

[fais3000]

@scottalanmiller So unfortunately none of the versions worked with our database any more. We even tried 1.11.2 and went back to 1.10.2 on mongodb 3.2 as well.

Now, we've updated mongodb to 3.4 and picked up an old database backup to migrate from 3.2 to 3.4 on the latest nodebb (1.11.2). Its working properly now.

However, a warning for people on older versions of mongodb when updating to 1.11.x onward.