@baris I'm really trying to stick to actual releases (looking at github it doesn't look like 1.17.1 has been officially blessed yet). However, staying on v1.17.0 and bumping the socket.io and socket.io-client versions seems to have done the trick!

I've been working through my custom plugins. I think I have all the breaking changes fixed but am still having this issue. So I deactivated them. Here's what I'm running now:

nodebb-plugin-composer-default
Installed 6.5.27 | Latest 6.5.27

nodebb-plugin-dbsearch
Installed 4.2.0 | Latest 4.2.0

nodebb-plugin-desktop-notifications
Installed 0.3.3 | Latest 0.3.3

nodebb-plugin-emoji
Installed 3.5.3 | Latest 3.5.3

nodebb-plugin-emoji-one
Installed 2.0.5 | Latest 2.0.5

nodebb-plugin-google-analytics
Installed 0.5.11 | Latest 0.6.0

nodebb-plugin-gravatar
Installed 2.3.0 | Latest 2.3.0

nodebb-plugin-iframely
Installed 0.7.14 | Latest 0.7.14

nodebb-plugin-imagemagick
Installed 2.1.1 | Latest 2.1.1

nodebb-plugin-markdown
Installed 8.12.7 | Latest 8.12.7

nodebb-plugin-mentions
Installed 2.13.9 | Latest 2.13.9

nodebb-plugin-shortcuts
Installed 1.1.2 | Latest 1.1.2

nodebb-plugin-spam-be-gone
Installed 0.7.9 | Latest 0.7.9

nodebb-plugin-sso-facebook
Installed 3.6.3 | Latest 3.6.3

nodebb-plugin-sso-github
Installed 2.3.1 | Latest 2.3.1

nodebb-plugin-sso-google
Installed 2.5.4 | Latest 2.5.4

nodebb-plugin-sso-twitter
Installed 2.5.11 | Latest 2.5.11

nodebb-rewards-essentials
Installed 0.1.4 | Latest 0.1.4

nodebb-theme-persona
Installed 11.0.17 | Latest 11.0.17

nodebb-widget-essentials
Installed 5.0.3 | Latest 5.0.3

@baris

[email protected] /home/boomzilla/nodebb
└── socket.io@4.0.1
└── [email protected]
└── [email protected]

New problem! Again, getting everything up to v1.17.0. We run clustered, so now we need redis (we run a postgresql DB for a DB). For background, in production we have docker images and blah blah blah...I've not been able to get that environment duplicated, so I'm just running stuff outside of docker.

In my dev VM I'm running Kubuntu 18.04. The "setting up redis" documentation was light on details like specific versions so I just installed the redis from my repo:

redis/bionic-updates,bionic-updates,bionic-security,bionic-security,now 5:4.0.9-1ubuntu0.2

Stuff mostly seems to work. However, when we're up and running if I click on an unread topic link this happens:

2021-05-12T23:25:07.092Z [4567/106387] - verbose: [plugins/fireHook] filter:topic.getFields
2021-05-12T23:25:07.092Z [4567/106387] - verbose: [plugins/fireHook] filter:privileges:isAllowedTo
2021-05-12T23:25:07.093Z [4567/106387] - verbose: [plugins/fireHook] filter:privileges:isAllowedTo
2021-05-12T23:25:07.093Z [4567/106387] - verbose: [plugins/fireHook] filter:privileges:isAllowedTo
2021-05-12T23:25:07.093Z [4567/106387] - verbose: [plugins/fireHook] filter:category.getFields
2021-05-12T23:25:07.093Z [4567/106387] - verbose: [plugins/fireHook] filter:privileges.topics.filter
2021-05-12T23:25:07.094Z [4567/106387] - verbose: [plugins/fireHook] filter:topic.getFields
2021-05-12T23:25:07.096Z [4567/106387] - verbose: [plugins/fireHook] filter:user.getSettings
2021-05-12T23:25:07.096Z [4567/106387] - verbose: [plugins/fireHook] filter:user.notificationTypes
2021-05-12T23:25:07.096Z [4567/106387] - verbose: [plugins/fireHook] filter:topics.getUnreadTids
2021-05-12T23:25:07.096Z [4567/106387] - verbose: [plugins/fireHook] filter:error.handle
2021-05-12T23:25:07.097Z [4567/106387] - error: /wtf/api/topic/13289/song-of-the-day/4062
TypeError: socket.client.writeToEngine is not a function
    at /home/boomzilla/nodebb/node_modules/socket.io-redis/node_modules/socket.io-adapter/dist/index.js:110:27
    at RedisAdapter.apply (/home/boomzilla/nodebb/node_modules/socket.io-redis/node_modules/socket.io-adapter/dist/index.js:194:25)
    at RedisAdapter.broadcast (/home/boomzilla/nodebb/node_modules/socket.io-redis/node_modules/socket.io-adapter/dist/index.js:109:14)
    at RedisAdapter.broadcast (/home/boomzilla/nodebb/node_modules/socket.io-redis/dist/index.js:321:15)
    at BroadcastOperator.emit (/home/boomzilla/nodebb/node_modules/socket.io/dist/broadcast-operator.js:109:22)
    at Topics.pushUnreadCount (/home/boomzilla/nodebb/src/topics/unread.js:254:44)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)
    at async Promise.all (index 1)
    at async markAsRead (/home/boomzilla/nodebb/src/controllers/topics.js:161:3)

Any idea what might be going on?

EDIT: If I refresh or hit enter from the address bar the page loads as expected.

@baris d'oh! I could have sworn that I'd updated them all to latest.

Thanks.

I'm working up to upgrading from 1.14.3 to 1.17.0 and working through breaking issues. I looked at the breaking issues for 1.15, 1.16 and 1.17 and I didn't see anything obvious.

First I had it with Github (v2.3.1)
https://github.com/julianlam/nodebb-plugin-sso-github

When I go to login, I got an error, so I removed that. Then I got the same error from v2.4.2:
https://github.com/julianlam/nodebb-plugin-sso-google

2021-05-12T17:31:56.251Z [4567/47491] - verbose: [plugins/fireHook] filter:middleware.buildHeader
(node:47491) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'push' of undefined
    at Object.Google.appendUserHashWhitelist [as method] (/home/boomzilla/tdwtf/plugins/nodebb-plugin-sso-google/library.js:110:18)
    at /home/boomzilla/nodebb/src/plugins/hooks.js:131:29
    at new Promise (<anonymous>)
    at fireMethod (/home/boomzilla/nodebb/src/plugins/hooks.js:130:10)
    at Object.fireFilterHook [as filter] (/home/boomzilla/nodebb/src/plugins/hooks.js:149:18)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)
    at async Hooks.fire (/home/boomzilla/nodebb/src/plugins/hooks.js:99:17)
    at async User.getUsersFields (/home/boomzilla/nodebb/src/user/data.js:59:19)
    at async User.bans.unbanIfExpired (/home/boomzilla/nodebb/src/user/bans.js:111:20)
    at async User.bans.canLoginIfBanned (/home/boomzilla/nodebb/src/user/bans.js:96:23)

Where do I go from here?

@pitaj hmm...looking more closely at this...updating the dependencies might do it. But if not, that one seems to use the Amazon published SDK so it should definitely work.

Thanks!

We've been using an Amazon SES plugin for mail. Recently got a message that we're using SES Signature v3 and we need to use v4. That plugin hasn't been updated since 2015.

Has anyone else solved this issue? Do we need to roll our own? Switch to a different service?

Our instance got upgraded (for reasons) from a 1.13 release to 1.14.0-7.

Would there be any danger in going back down to the last stable release in 1.13? I guess I'm concerned about any potential DB "schema" changes that might have occurred when we upgraded to 1.14.

[edited to fix version numbers...Fridays, eh?]

This was reported:
https://github.com/boomzillawtf/nodebb-plugin-youtube-embed/issues/9

I can't reproduce, but the report says they get an error when navigating to the settings in the admin area (NodeBB 1.7.5, youtube-embed v0.6.7). Does anyone have an idea about what's going on here?

Unable to load template: admin/plugins/youtube-lite
(anonymous) @ ajaxify.js:333
(anonymous) @ require.js:7
(anonymous) @ require.js:23
y @ require.js:6
emit @ require.js:23
(anonymous) @ require.js:11
y @ require.js:6
v @ require.js:11
onScriptError @ require.js:30
require.js:7 Uncaught Error: Script error for "/assets/templates/admin/plugins/youtube-lite.js"
http://requirejs.org/docs/errors.html#scripterror
    at F (require.js:7)
    at HTMLScriptElement.onScriptError (require.js:30)
F @ require.js:7
onScriptError @ require.js:30
ajaxify.js:333 Unable to load template: admin/plugins/youtube-lite
(anonymous) @ ajaxify.js:333
(anonymous) @ require.js:7
(anonymous) @ require.js:23
y @ require.js:6
emit @ require.js:23
w @ require.js:12
(anonymous) @ require.js:12
y @ require.js:6
w @ require.js:12
(anonymous) @ require.js:13
y @ require.js:6
C @ require.js:13
(anonymous) @ require.js:13
setTimeout (async)
C @ require.js:13
(anonymous) @ require.js:13
setTimeout (async)
C @ require.js:13
(anonymous) @ require.js:13
setTimeout (async)
C @ require.js:13
(anonymous) @ require.js:13
setTimeout (async)
C @ require.js:13
(anonymous) @ require.js:13
setTimeout (async)
C @ require.js:13
(anonymous) @ require.js:13
setTimeout (async)
C @ require.js:13
(anonymous) @ require.js:13
setTimeout (async)
C @ require.js:13
(anonymous) @ require.js:13
setTimeout (async)
C @ require.js:13
(anonymous) @ require.js:13
setTimeout (async)
C @ require.js:13
(anonymous) @ require.js:13
setTimeout (async)
C @ require.js:13
(anonymous) @ require.js:13
setTimeout (async)
C @ require.js:13
(anonymous) @ require.js:13
setTimeout (async)
C @ require.js:13
(anonymous) @ require.js:13
setTimeout (async)
C @ require.js:13
(anonymous) @ require.js:26
setTimeout (async)
g.nextTick @ require.js:32
a @ require.js:26
requirejs @ require.js:32
ajaxify.loadTemplate @ ajaxify.js:332
(anonymous) @ benchpress.js:163
a @ benchpress.js:162
(anonymous) @ benchpress.js:184
(anonymous) @ benchpress.js:77
e @ benchpress.js:76
l @ benchpress.js:183
e @ benchpress.js:222
d @ ajaxify.js:185
(anonymous) @ ajaxify.js:98
success @ ajaxify.js:317
l @ jquery.js:3268
fireWith @ jquery.js:3398
c @ jquery.js:9305
(anonymous) @ jquery.js:9548
XMLHttpRequest.send (async)
(anonymous) @ VM7656:1
send @ jquery.js:9600
ajax @ jquery.js:9206
ajaxify.loadData @ ajaxify.js:291
ajaxify.go @ ajaxify.js:86
n @ ajaxify.js:362
(anonymous) @ ajaxify.js:413
dispatch @ jquery.js:5183
g.handle @ jquery.js:4991
require.js:7 Uncaught Error: Script error for "/assets/templates/admin/plugins/youtube-lite.js"
http://requirejs.org/docs/errors.html#scripterror
    at F (require.js:7)
    at HTMLScriptElement.onScriptError (require.js:30)

@baris said in Bookmarks / Favourites breaking change 1.2:

Actually we do breaking changes in minor releases and not patch releases, so we don't really follow semver in that regard.

Ah, fair enough.

@pichalite said in Bookmarks / Favourites breaking change 1.2:

Also 1.3 is not released yet.

I know. But it seems like an awful way to run a railroad if you break people's plugins without advance notice, which I don't think is covered by a tag on a github issue.

Recently, a breaking change (1.2 -> 1.3) was introduced:

refactor src/favourites.js · Issue #5096 · NodeBB/NodeBB

rename uid::favourites to uid::bookmarks rename pid::users_favourited to pid::users_bookmarked rename /favourites route to /bookmarks rename post field reputation to bookmarks separate vote and bookmark code fix langu...

GitHub (github.com)

I didn't see any sort of announcement or warning about this (though admittedly I wasn't watching terribly closely). Was there a public discussion about this? I'm curious as to why you guys introduced this breaking change in a minor release.

Thanks.

Here's a thing. Had a nodebb process pegging the CPU and ran the perf tool against it. It collected 2,967 identical stacks:

node 29869 cpu-clock: 
            3288d1761413 LazyCompile:~ /usr/src/app/node_modules/nodebb-plugin-mentions/library.js:165 (/tmp/perf-29869.map)
            3288d1772468 LazyCompile:~ /usr/src/app/node_modules/nodebb-plugin-mentions/library.js:239 (/tmp/perf-29869.map)
            3288d1770e25 LazyCompile:~ /usr/src/app/node_modules/nodebb-plugin-mentions/library.js:209 (/tmp/perf-29869.map)
            3288d135f1f9 LazyCompile:~ /usr/src/app/src/database/mongo/sorted.js:341 (/tmp/perf-29869.map)
            3288d179b3fa LazyCompile:* /usr/src/app/src/topics/unread.js:340 (/tmp/perf-29869.map)
            3288d15bd907 LazyCompile:*readableAddChunk _stream_readable.js:124 (/tmp/perf-29869.map)
            3288d13e12ce LazyCompile:*Readable.push _stream_readable.js:100 (/tmp/perf-29869.map)
            7f52c6fdbb45 init_cacheinfo (/lib/x86_64-linux-gnu/libc-2.19.so)

So now the mentions plugin is suspect #1. cc: @julianlam

@torn2 iFramely has some problems with preview. If you put something in there and then start typing, you basically spam whatever server. Users really hated that, so at least for youtube we went with this.

Re: [nodebb-plugin-youtube-lite] Server side modifications

I forked nodebb-plugin-youtube-lite with my enhancements.

nodebb-plugin-youtube-embed

NodeBB Youtube Embed Plugin. Latest version: 0.6.9, last published: 4 years ago. Start using nodebb-plugin-youtube-embed in your project by running `npm i nodebb-plugin-youtube-embed`. There are no other projects in the npm registry using nodebb-plugin-youtube-embed.

npm (www.npmjs.com)

We're running our instance with --perf-basic-prof-only-functions now and are able to generate flamegraphs. Just waiting for the servercooties to strike now...

Example flamegraph:

@torn2 Not to my knowledge. It works just like using their service except it all happens on your server (or servers, if you put it on a separate machine).

We see similar issues. Sometimes we have to restart many times per day, some days it will keep going. we've been tracking it here:

• https://what.thedailywtf.com/topic/19990/the-official-server-cooties-tracking-thread

We're running stuff under docker, which makes things more complicated, but I've had luck in a local dev/test environment with using Linux perf tools and building flamegraphs. For raisins, we haven't done this yet in docker, though you can see how to make that work here.

However, if you're not running docker, it's a little bit easier. Follow the instructions here:

• https://yunong.io/2015/11/23/generating-node-js-flame-graphs/

He doesn't go into it there, but if you're on Ubuntu you'll need to install:

• linux-tools-common
• linux-cloud-tools-common

...and then kernel specific versions of those based on what you're running in order to use perf.

If you get there before we do, I'm eager to hear what anyone discovers with this.

@a_5mith are you still there? Maybe it's time to fork...

@exodo Just put it in:

Server side by boomzillawtf · Pull Request #30 · a5mith/nodebb-plugin-youtube-lite

These changes simplify the client handling. The API calls are all done on the server (and cached with an LRU to minimize quota usage) with graceful degradation without an API key. Above the video, ...

GitHub (github.com)