How to restore topic posters as topic followers?

Solved Technical Support
  • After importing forums into NodeBB, users are not getting notifications of new replies to topics they have replied to. They are not set as followers/watchers of their topics anymore, excepting the first poster only.

    An example of a MongoDB record for a long topic with a large number of posters but only three followers (OP, admin, tester):

    {
        "_id" : ObjectId("5ab6da748fda465c4283b0f5"),
        "_key" : "tid:899:followers",
        "members" : [ 
            "4614", 
            "1", 
            "20790"
        ]
    }
    

    The follower with uid 4614 is the original poster of the topic. 1 is of course the admin. 20790 is a temporary user to test adding himself manually as watcher of the topic.

    Related to this record, there are also a large number of records for the many posters on this long forum topic (e.g., for users 4614, 9623...):

    {
        "_id" : ObjectId("5ab6da748fda465c4283b0e5"),
        "_key" : "tid:899:posters",
        "value" : "4614",
        "score" : 8
    }
    
    {
        "_id" : ObjectId("5ab6dc328fda465c4292cfff"),
        "_key" : "tid:899:posters",
        "value" : "9623",
        "score" : 1
    }
    
    etc...
    

    This happens with all topics and users of the imported forums.

    A good way to fix it would be if selecting "Follow topics that you reply to" as default user setting configured existing users as well, since that's usually the expected behavior for forums. However, currently this setting doesn't restore previous topic posters as topic followers, and they don't receive notifications.

    Please, is there a way to restore them as followers? Those forums have thousands of users, so we can't manually edit the databases. Maybe some kind of script for MongoDB? Not sure how to fix it.

  • @juan-g Here is a one time script to get the posters of each topic and have them follow that topic. Make sure you take a backup of your database before running this as I haven't tested it on a live database. Hope it helps.

    /*globals require, console, process */
    'use strict';
    
    var nconf = require('nconf');
    var async = require('async');
    
    nconf.file({
    	file: 'config.json'
    });
    
    nconf.defaults({
    	base_dir: __dirname,
    });
    
    var db = require('./src/database');
    
    db.init(function(err) {
    	if (err) {
    		console.log("NodeBB could not connect to your database. Error: " + err.message);
    		process.exit();
    	}
    
    	followTopics(function (err) {
    		if (err) {
    			console.error(err);
    			process.exit();
    		}
    		console.log('done');
    		process.exit();
    	});
    });
    
    function followTopics(callback) {
    	db.getSortedSetRange('topics:tid', 0, -1, function (err, tids) {
    		if (err) {
    			return callback(err);
    		}
    		
    		async.eachSeries(tids, followTopicByPosters, callback);
    	});
    }
    
    function followTopicByPosters(tid, callback) {
    	db.getSortedSetRange('tid:' + tid + ':posters', 0, -1, function (err, uids) {
    		if (err) {
    			return callback(err);
    		}
    
    		if (!uids.length) {
    			return callback();
    		}
                    db.setAdd('tid:' + tid + ':followers', uids, callback);
    	});
    }
    

    You have to place this in your NodeBB root folder and run it with node myScript.js

  • @baris Wow, thank you, that's amazing. A doubt before running it. As said, the first poster of each topic is already a follower (although not the rest of posters), however I think db.setAdd will prevent duplicate uids when adding them this way, to avoid maybe duplicate notifications for the OPs. But I'm not sure. Is that so? Thanks!

  • @juan-g yes db.setAdd won't add duplicates.

  • @baris Almost there, this is really wonderful. I did, from the nodebb folder:

    ./nodebb stop
    
    mongodump -u nodebb -p [password] -d nodebb -o dump/nodebb-2018-05-03
    
    node restoreTopicFollowers.js
    

    Then verified the MongoDB database (I use Robo 3T for that), e.g.:

    db.getCollection('objects').find({_key: /:followers/})
    

    and indeed the followers had been correctly added to each topic on the database!

    So I did:

    ./nodebb start
    

    And testing NodeBB there is a final glitch: Those added followers still show "This user hasn't watched any topics yet" in their "Watched" sections. I've tried rebuilding and restarting NodeBB, and deleting browser cache, but no change there.

    Is there anything I can try for that?

  • Oh knew I forgot something, run the below to create those sets.

    /*globals require, console, process */
    'use strict';
    
    var nconf = require('nconf');
    var async = require('async');
    
    nconf.file({
    	file: 'config.json'
    });
    
    nconf.defaults({
    	base_dir: __dirname,
    });
    
    var db = require('./src/database');
    
    db.init(function(err) {
    	if (err) {
    		console.log("NodeBB could not connect to your database. Error: " + err.message);
    		process.exit();
    	}
    
    	followTopics(function (err) {
    		if (err) {
    			console.error(err);
    			process.exit();
    		}
    		console.log('done');
    		process.exit();
    	});
    });
    
    function followTopics(callback) {
    	db.getSortedSetRange('topics:tid', 0, -1, function (err, tids) {
    		if (err) {
    			return callback(err);
    		}
    
    		async.eachSeries(tids, followTopicByPosters, callback);
    	});
    }
    
    function followTopicByPosters(tid, callback) {
    	db.getSortedSetRange('tid:' + tid + ':posters', 0, -1, function (err, uids) {
    		if (err) {
    			return callback(err);
    		}
    
    		if (!uids.length) {
    			return callback();
    		}
    
    		var keys = uids.map(function (uid) {
    			return 'uid:' + uid + ':followed_tids';
    		});
    		db.sortedSetsAdd(keys, now, tid, callback);
    	});
    }
    
  • @baris It returns an error now is not defined for the line 56:

    db.sortedSetsAdd(keys, now, tid, callback);
    
  • Maybe I can try using Date.now() instead of now, but I don't know if that's right for this.

  • This is great. It works! 🎆

    So, if I'm not wrong, to fix other imported forums we should also run both scripts (restoreTopicFollowers.js and restoreTopicFollowers2.js), the second one with the minor replacement now -> Date.now()

    Really, thank you very much! 👍

  • Yes, typically when you see now used in our code there's probably a var now = Date.now(); nearby 😆

  • @juan-g Yeah you can declare now above the code with var now = Date.now()


Suggested Topics


  • 0 Votes
    1 Posts
    153 Views

    Hi.

    The page Recent at a forum which uses NodeBB shows only 10 pages of "recent" topics.
    When I am at the 10th page, the ">" sign at the bottom of the page isn't clickable (has no link).

    I even tried to manually go to the next page by means of typing https://thedomain/forum/recent?page=11, but it returns

    There are no recent topics.

    I want to be able to read older topics because I'm trying to catch up (during a few last months I've been reading that forum quite rarely).

    Is it possible for the forum admin to enable showing all previous topics on the Recent page?
    Thank you in advance!

  • how to add HTML content in topics

    Solved Technical Support
    0 Votes
    2 Posts
    2k Views

    Use of the Markdown composer is optional, and you can save HTML directly into the database, and this is what is used for other composers like Redactor. Still other composers (Quill composer) save content into the database in their own special format.

    Markdown is not meant to be activated for either of those.

    Alternatively you could allow HTML in the markdown settings, but I would highly advise you also install sanitizehtml plugin, otherwise anyone can post anything including malicious scripts as a post in your forum!

  • 0 Votes
    3 Posts
    988 Views

    @jimmyc2018 maybe your server restarted without you knowing and mongo and nodebb didn't automatically start back up?
    Have you tried starting the mongod service and then starting nodebb?

    Btw, you shouldn't run NodeBB with root or sudo. That may also be causing the problem.

  • 0 Votes
    21 Posts
    7k Views

    @Fastidious hehe I did feel like I was using a sledgehammer to crack a nut at the time.

  • 0 Votes
    1 Posts
    1k Views

    Hi all,

    We are currently looking for a way to customise the Lavender theme to make the user's Avatar/profile section in Topic View more "gamified" by showing statistics as well as the "Experience Bar" from the ns-points plugin.

    Currently, the theme looks like this when a user posts in a thread:

    ERROR: The request could not be satisfied

    favicon

    (cdn.experienceoz.com.au)

    but I'd like it to look more like:

    ERROR: The request could not be satisfied

    favicon

    (cdn.experienceoz.com.au)

    to help show progression, be more visual, etc. Is it possible to customise this through one of the .tpl files or would it require a root change to how the plugin works? Our sys admin says he thought it would require a change to the actual plugin; I thought it might be a matter of just relocating some of the CSS classes to the "profile.tpl" section of the user profile.

    Any help or feedback would be greatly appreciated, cheers 🙂