3.3.0 Upgrade Support
-
Just to rule out that it's the dependency upgrade you can try to use the versions from 3.2.3 and try to run the upgrade script.
Install pg & pg-cursor that 3.2.3 uses with the below commands than try to run the upgrade script again.
npm i [email protected] npm i [email protected]
-
@baris said in 3.3.0 Upgrade Support:
npm i [email protected]
still stalling...
4. Updating NodeBB data store schema... Parsing upgrade scripts... OK | 1 script(s) found, 129 skipped → [2023/7/27] Set mid on message objects and create messages:mid... [##############] (16/16) 100%
-
I am getting this error when upgrading:
Parsing upgrade scripts... OK | 5 script(s) found, 124 skipped → [2023/7/2] Update chat messages to add roomId field... Error occurred [##############] (401/401) 100% 2023-08-17T20:42:31.417Z [4567/9] - warn: NodeBB Setup Aborted. MongoServerError: error while multiplanner was selecting best plan :: caused by :: Sort exceeded memory limit of 104857600 bytes, but did not opt in to external sorting. at Connection.onMessage (/usr/src/app/nodebb/node_modules/mongodb/lib/cmap/connection.js:202:26) at MessageStream.<anonymous> (/usr/src/app/nodebb/node_modules/mongodb/lib/cmap/connection.js:61:60) at MessageStream.emit (node:events:513:28) at processIncomingData (/usr/src/app/nodebb/node_modules/mongodb/lib/cmap/message_stream.js:124:16) at MessageStream._write (/usr/src/app/nodebb/node_modules/mongodb/lib/cmap/message_stream.js:33:9) at writeOrBuffer (node:internal/streams/writable:392:12) at _write (node:internal/streams/writable:333:10) at Writable.write (node:internal/streams/writable:337:10) at Socket.ondata (node:internal/streams/readable:766:22) at Socket.emit (node:events:513:28)
Anyone else seen this issue ?
-
@baris said in NodeBB 3.3.0:
@fgallese do you have a room with a lot of users by any chance? Or were you using nodebb-plugin-global-chat?
Yes and yes. We have a chat room with all users which sees a lot of usage, so it is expected for us to have a lot of messages in the history I guess.
-
@baris said in NodeBB 3.3.0:
Hmm in that case I will have to modify that upgrade script so it doesn't load all uids/messageIds in the room.
Do you think there is any quick workaround I can apply in my db to help this ? Like message trimming or similar ?
-
You could try adding
.allowDiskUse(true)
to the query at https://github.com/NodeBB/NodeBB/blob/master/src/database/mongo/sorted.js#L397-L401.It throws that error because the data is too big to sort in memory (the default limit is 100mb).
I am working on the new upgrade script that should be aviable later today as 3.3.1.
-
@fgallese can you try replacing
src/upgrades/3.3.0/chat_room_refactor.js
with the new one from https://github.com/NodeBB/NodeBB/blob/master/src/upgrades/3.3.0/chat_room_refactor.js.After that try running it again, it will run slower but shouldn't cause the memory limit issue.
-
@baris It can be successful here. On my site, a breakpoint error is reported in fetch, which seems to be the reason for packaging.
I replaced fetch with $.ajax and restored
async function xhr(options, cb) { // Allow options to be modified by plugins, etc. ({ options } = await fireHook('filter:api.options', { options })); $.ajax(options) .done((res) => { cb(null, ( res && res.hasOwnProperty('status') && res.hasOwnProperty('response') ? res.response : (res || {}) )); }) .fail((ev) => { let errMessage; if (ev.responseJSON) { errMessage = ev.responseJSON.status && ev.responseJSON.status.message ? ev.responseJSON.status.message : ev.responseJSON.error; } cb(new Error(errMessage || ev.statusText)); }); }
-
@陈洁 Looks like you modified the code and added try/catch around the fetch which is causing the
res
variable to be undefined further down the code.It should be
const res = await fetch(url, options); const { headers } = res; const contentType = headers.get('content-type'); const isJSON = contentType && contentType.startsWith('application/json'); let response; if (isJSON) { response = await res.json(); } else { response = await res.text(); } if (!res.ok) { return cb(new Error(isJSON ? response.status.message : response)); }