[Solved] Wrong types in redis database

geoffb

Quick background: I had some issues upgrading my NodeBB install a month or so a ago.

I'm currently running 2.1 but trying to upgrade to the latest 2.x (and following the upgrade procedures this time!). However, I seem to be running into an issue where the types of certain keys within redis are incorrect. For example, the latest 2.x code expects the key "tid:22:posts" to be a list, but it's a zset in my database.

There aren't any errors during the database upgrade, but the server crashes trying to fetch the zset data using lrange. It's not limited just to the tid:* keys, it seems there are several keys that are zsets when they should be lists.

I'm looking for suggestions on how to fix these data types while retaining the current data. A quick google for converting zsets to lists didn't turn up much of anything. I'm pretty new to redis in general, but I'm able to inspect and modify the data via command line or redis-commander. Any help is appreciated!

<baris>

Hi @geoffb ,

That sounds weird since the upgrade converts those lists to zsets and the accompanying code should be using zrange.

The part of the upgrade script that converts those lists into zsets is this

function upgradeTopicPosts(next) {
				function upgradeTopic(tid, next) {
					function addPostToTopic(pid) {
						Posts.getPostField(pid, 'timestamp', function(err, timestamp) {
							db.sortedSetAdd('tid:' + tid + ':posts', timestamp, pid);
						});
					}

					db.getListRange('tid:' + tid + ':posts', 0, -1, function(err, pids) {
						if(err) {
							return next(err);
						}

						if(!pids || !pids.length) {
							return next();
						}

						db.delete('tid:' + tid + ':posts', function(err) {
							for(var i = 0; i< pids.length; ++i) {
								addPostToTopic(pids[i]);
							}
							next();
						});
					});
				}

				db.getSetMembers('topics:tid', function(err, tids) {
					async.each(tids, upgradeTopic, function(err, results) {
						next(err);
					});
				});
			}

It reads from the list deletes it and adds them back as a sorted set. I am assuming this has ran successfully for you since you say tid:22:posts is a sorted set. Does it work for you when you switch to 0.3.0 branch? As always make a backup of your database. Tagging @julian.

geoffb

I've tried upgrading to v0.3.x and I get the following error while running the upgrade script:

info: Beginning database schema update
error: [upgrade] Errors were encountered while updating the NodeBB schema: Error: WRONGTYPE Operation against a key holding the wrong kind of value

I'm testing these upgrades on a copy of the database running locally.

<baris>

So its failing on the first upgrade which is

db.getListRange('categories:cid', 0, -1, function(err, cids) {

If your categories are already a sorted set you can remove that block.

geoffb

Thanks for the help here, @baris.

So, categories:cid is a list and the db.getListRange command seems to run correctly. The error occurs in the upgradeCategory function while attempting to run db.getSetMembers('cid:' + cid + ':active_users'.

Running TYPE cid:10:active_users returns "zset".

<baris>

You can change

db.getSetMembers('cid:' + cid + ':active_users', function(err, uids) {
	if(err) {
		return next(err);
    }

to

db.getSetMembers('cid:' + cid + ':active_users', function(err, uids) {
	if(err) {
		return next();
	}

To ignore that error.

geoffb

I skipped the category upgrade, the "user topic and post upgrade to sorted set" section, and now the v0.3.x upgrade completes without errors. The forum loads and so far I haven't seen any crashes while browsing around. Looks like we might be in good shape. I'm guessing that my original database botching left some of the 0.3.x upgrades intact even though I was running on 0.2.1.

As always, thanks for the help!

<baris>

No problem! Let us know if anything is not working.

julian

Glad you got it resolved @geoffb

Thanks @baris for triaging...