Total vote count on topic list

crazycells

@phenomlab yeap, we have both cases in the forum, but this calculation makes more sense to us than the first post only.

phenomlab

@crazycells said in Total vote count on topic list:

but this calculation makes more sense to us than the first post only.

And for me.

crazycells

hi @baris and @oplik0 , "most votes" ranking option is not respecting this plugin, what would be the easiest way to solve this, so the topics are ranked according to the total vote counts rather than the vote count of the first post...

Screenshot 2024-03-16 at 23.41.11.png

phenomlab

@baris @oplik0 any thoughts in relation to the above post?

<baris>

The plugin would need to be updated to create a new sorted set with the new scores(total votes). Not exactly a simple change.

phenomlab

@baris is there any appetite to do this? If not, I will need to uninstall the total votes plugin...

<baris>

@phenomlab I sent a PR to @oplik0 's repro that will change the sorting. https://github.com/oplik0/nodebb-plugin-total-vote-count/pull/19

Just a warning, this will only apply to topics when they receive a new vote on one of their posts. It basically recalculates the new order whenever there is a new vote. Also once this plugin is active uninstalling it will not magically fix the scores to go back to the old sorting. You would need a custom script to fix all the topic votes to set them back to the main post only.

phenomlab

@baris Ok. Many thanks

crazycells

thanks a lot @baris

I Really Appreciate You Curtis Payne GIF

just a quick question, this plugin is not causing any performance issues, right? I am not planning to uninstall it since it is -for us- more correct way to calculate the topic vote counts, but wanted to check since you mentioned some side effects of the plugin after uninstalling it...

<baris>

It might slow down the topic list if there are topics with a huge amount of posts, since it is calculating the total vote count when you load it. You would have to test and see if the performance is acceptable.

crazycells

@baris theoretically, this action will only happen when we try to rank the topics according to votes, right? I mean, if recently replied is picked, should this create a performance issue?

in our forum, most replied topics have anywhere between 20K-30K posts, not more than that...

<baris>

No the code in the plugin runs whenever you load a list of topics to calculate the votes count displayed. Instead of displaying the normal value it is calculating total votes from each post in the topic. So yeah topics with 30k posts might slow things down but I haven't tested it.

phenomlab

@baris my concern at this point is related to performance. Especially if things remain the same even after this specific plugin is disabled or uninstalled.

Will there be an option in the admin console to reset this with the script you previously mentioned?

<baris>

@phenomlab you can run below script in your nodebb folder to reset to the old scores.

/* eslint-disable no-await-in-loop */
/* globals require, console, process */

'use strict';

const nconf = require('nconf');

nconf.file({
	file: 'config.json',
});

nconf.defaults({
	base_dir: __dirname,
	views_dir: './build/public/templates',
	upload_path: 'public/uploads',
});

const db = require('./src/database');

db.init(async (err) => {
	if (err) {
		console.log(`NodeBB could not connect to your database. Error: ${err.message}`);
		process.exit();
	}

	await fixVotes();
	console.log('done');
	process.exit();
});


async function fixVotes() {
        const batch = require('./src/batch');
        await batch.processSortedSet('topics:tid', async (tids) => {
                let topicData = await db.getObjectsFields(
                        tids.map(tid => `topic:${tid}`),
                        ['tid', 'cid', 'timestamp', 'upvotes', 'downvotes']
                );
                topicData = topicData.filter(t => t && t.cid);
                topicData.forEach((t) => {
                        t.votes = parseInt(t.upvotes, 10) - parseInt(t.downvotes, 10);
                });

                await db.sortedSetAddBulk(
                        topicData.map(t => ([`cid:${t.cid}:tids:votes`, t.votes, t.tid]))
                );
                await db.sortedSetAddBulk(
                        topicData.map(t => ([`topic:votes`, t.votes, t.tid]))
                );

                
        }, {
                batch: 500,
        });
}

phenomlab

@baris thanks

crazycells

@baris said in Total vote count on topic list:

No the code in the plugin runs whenever you load a list of topics to calculate the votes count displayed. Instead of displaying the normal value it is calculating total votes from each post in the topic. So yeah topics with 30k posts might slow things down but I haven't tested it.

yes, unfortunately, there is a performance issue... there is several seconds of delay while topics are listed...

phenomlab

@crazycells said in Total vote count on topic list:

there is a performance issue...

Which is why I want to back this out as I'm experiencing a similar issue. I wonder if it's possible to calculate the total votes as a nightly routine which updates a table (for example). That table is referenced on topic list load so topic id = total votes, etc.

A nighttime run that can be specified as off hours would work well in this case. Not realtime but certainly usable. Plus the ability to run the task adhoc in the admin console....

Just a thought.

crazycells

@phenomlab said in Total vote count on topic list:

@crazycells said in Total vote count on topic list:

there is a performance issue...

Which is why I want to back this out as I'm experiencing a similar issue. I wonder if it's possible to calculate the total votes as a nightly routine which updates a table (for example). That table is referenced on topic list load so topic id = total votes, etc.

A nighttime run that can be specified as off hours would work well in this case. Not realtime but certainly usable. Plus the ability to run the task adhoc in the admin console....

Just a thought.

I totally agree with you. I definitely think this is the correct way of calculation topic vote counts...

I believe it is a dynamic function that makes the calculations each time a topic list page is loaded... however it does not have to be that dynamic... It would be great, if what you said can be applied, a list with these variables can be re-calculated every 24 hours... Ultimately, topics are not typically ranked with votes 345, 346 and 347 anyways... instead, they would be something like one with 300s, another one with 400s, one with 600s... So, it does not need to be an exact number at any given moment, daily fluctuations should not significantly affect the rankings...

phenomlab

@baris said in Total vote count on topic list:

Also once this plugin is active uninstalling it will not magically fix the scores to go back to the old sorting

Not the case in my experience. I uninstalled the plugin, and everything returned to normal - I did not need to do anything /top now works as expected, but would prefer this to be the total votes - obviously, unless we can run this as a scheduled task as I previously mentioned, then it's non-starter because of the performance impact.
I did try to run the script, but it didn't work

/home/sudonix.org/nodebb/vote.js:53
                progress: this.progress,
                               ^

TypeError: Cannot read properties of undefined (reading 'progress')
    at fixVotes (/home/sudonix.org/nodebb/vote.js:53:32)
    at /home/sudonix.org/nodebb/vote.js:26:8
    at /home/sudonix.org/nodebb/src/promisify.js:43:65
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)

<baris>

@phenomlab I fixed the script in the above post, also made some changes to the plugin so it doesn't run the calculation on every page load. https://github.com/oplik0/nodebb-plugin-total-vote-count/pull/21

It will only calculate the total vote count if it is missing so any loads after that will use the cached value. Hopefully that improves performance.