3.3.0 Upgrade Support

sweetp

$ npm ls pg
[email protected] /var/www/vhosts/…/NodeBB
├─┬ [email protected]
│ └── [email protected] deduped
├─┬ [email protected]
│ └── [email protected] deduped
└─┬ [email protected]
  └─┬ [email protected]
    └── [email protected] deduped

$ npm ls pg-cursor
[email protected] /var/www/vhosts/…/NodeBB
└── [email protected]

$ npm ls pg
[email protected] /var/www/vhosts/…/NodeBB
├─┬ [email protected]
│ └── [email protected] deduped
├─┬ [email protected]
│ └── [email protected] deduped
└─┬ [email protected]
  └─┬ [email protected]
    └── [email protected] deduped

$ npm ls pg-cursor
[email protected] /var/www/vhosts/…/NodeBB
└── [email protected]

<baris>

Just to rule out that it's the dependency upgrade you can try to use the versions from 3.2.3 and try to run the upgrade script.

Install pg & pg-cursor that 3.2.3 uses with the below commands than try to run the upgrade script again.

npm i [email protected]
npm i [email protected]

sweetp

@baris said in 3.3.0 Upgrade Support:

npm i [email protected]

still stalling...

4. Updating NodeBB data store schema...
Parsing upgrade scripts...
OK | 1 script(s) found, 129 skipped
  → [2023/7/27] Set mid on message objects and create messages:mid...
    [##############] (16/16) 100%

<baris>

I will give it a shot with psql 15 and see if I can reproduce that way.

<baris>

@sweetp works for me on psql15 as well. What version of nodejs are you using?

sweetp

@baris Node v20.5.1

<baris>

I was on 18.x, tested on 20.5.1 works there too, although I found this which sounds similar https://github.com/brianc/node-postgres/issues/2069

fgallese

I am getting this error when upgrading:

Parsing upgrade scripts... 
OK | 5 script(s) found, 124 skipped
  → [2023/7/2] Update chat messages to add roomId field...
Error occurred
    [##############] (401/401) 100% 2023-08-17T20:42:31.417Z [4567/9] - warn: NodeBB Setup Aborted.
 MongoServerError: error while multiplanner was selecting best plan :: caused by :: Sort exceeded memory limit of 104857600 bytes, but did not opt in to external sorting.
    at Connection.onMessage (/usr/src/app/nodebb/node_modules/mongodb/lib/cmap/connection.js:202:26)
    at MessageStream.<anonymous> (/usr/src/app/nodebb/node_modules/mongodb/lib/cmap/connection.js:61:60)
    at MessageStream.emit (node:events:513:28)
    at processIncomingData (/usr/src/app/nodebb/node_modules/mongodb/lib/cmap/message_stream.js:124:16)
    at MessageStream._write (/usr/src/app/nodebb/node_modules/mongodb/lib/cmap/message_stream.js:33:9)
    at writeOrBuffer (node:internal/streams/writable:392:12)
    at _write (node:internal/streams/writable:333:10)
    at Writable.write (node:internal/streams/writable:337:10)
    at Socket.ondata (node:internal/streams/readable:766:22)
    at Socket.emit (node:events:513:28)

Anyone else seen this issue ?

<baris>

@fgallese do you have a room with a lot of users by any chance? Or were you using nodebb-plugin-global-chat?

fgallese

@baris said in NodeBB 3.3.0:

@fgallese do you have a room with a lot of users by any chance? Or were you using nodebb-plugin-global-chat?

Yes and yes. We have a chat room with all users which sees a lot of usage, so it is expected for us to have a lot of messages in the history I guess.

<baris>

Hmm in that case I will have to modify that upgrade script so it doesn't load all uids/messageIds in the room.

fgallese

@baris said in NodeBB 3.3.0:

Hmm in that case I will have to modify that upgrade script so it doesn't load all uids/messageIds in the room.

Do you think there is any quick workaround I can apply in my db to help this ? Like message trimming or similar ?

<baris>

You could try adding .allowDiskUse(true) to the query at https://github.com/NodeBB/NodeBB/blob/master/src/database/mongo/sorted.js#L397-L401.

It throws that error because the data is too big to sort in memory (the default limit is 100mb).

I am working on the new upgrade script that should be aviable later today as 3.3.1.

陈洁

Sign up button doesn't work,Will not jump after registration v3.3.0

<baris>

@fgallese can you try replacing src/upgrades/3.3.0/chat_room_refactor.js with the new one from https://github.com/NodeBB/NodeBB/blob/master/src/upgrades/3.3.0/chat_room_refactor.js.

After that try running it again, it will run slower but shouldn't cause the memory limit issue.

<baris>

@陈洁 Can you reproduce on this forum? We are running 3.3.0 here as well.

陈洁

@baris It can be successful here. On my site, a breakpoint error is reported in fetch, which seems to be the reason for packaging.

I replaced fetch with $.ajax and restored

async function xhr(options, cb) {
	// Allow options to be modified by plugins, etc.
	({ options } = await fireHook('filter:api.options', { options }));

	$.ajax(options)
		.done((res) => {
			cb(null, (
				res &&
				res.hasOwnProperty('status') &&
				res.hasOwnProperty('response') ? res.response : (res || {})
			));
		})
		.fail((ev) => {
			let errMessage;
			if (ev.responseJSON) {
				errMessage = ev.responseJSON.status && ev.responseJSON.status.message ?
					ev.responseJSON.status.message :
					ev.responseJSON.error;
			}

			cb(new Error(errMessage || ev.statusText));
		});
}

陈洁

@baris Now it runs successfully. I don’t know why fetch can’t run. On my website

<baris>

@陈洁 What is the value of url in your config.json? Try removing the www if you have it there.

<baris>

@陈洁 Looks like you modified the code and added try/catch around the fetch which is causing the res variable to be undefined further down the code.

It should be

const res = await fetch(url, options);
const { headers } = res;
const contentType = headers.get('content-type');
const isJSON = contentType && contentType.startsWith('application/json');

let response;
if (isJSON) {
	response = await res.json();
} else {
	response = await res.text();
}

if (!res.ok) {
	return cb(new Error(isJSON ? response.status.message : response));
}