Disqus importer

sanatisharif

@bentael sorry about that,
We actually have a topic for each Disqus thread in NodeBB right now. We want to add those Disqus posts to the existing topics we have. Does it answer your question?

sanatisharif

@bentael
First of all I think it is a good idea to mention that we are using NodeBB as our forum and also comment system.I mean by using comment plugin we have published our webpages on our forum so we have a forum topic for each page.
Here is my suggestion for this plugin:
The plugin could have a section for uploading Disqus entries file . After uploading the file the plugin will recognize Disqus threads and will show me a table like below:

0_1452513696214_screenshot-www.tablesgenerator.com 2016-01-11 15-31-26.png

As you see this table contains my Disqus threads' title and link and a blank column for putting the suitable NodeBB Topic ID of the thread. So after filling in this table and pressing on a submit , Disqus posts will be inserted to desirable topics. Forgive me for pointing out two issues here:
a. Two or more threads can have same topics
b. Similar to NodeBB posts , Disqus posts have insertion time, so during the merging process the time issue should be taken care of .

So do you think it can be done?
Thanks a lot

bentael

@sanatisharif said:

We actually have a topic for each Disqus thread in NodeBB right now. We want to add those Disqus posts to the existing topics we have. Does it answer your question?

yes, that makes more sense

So after filling in this table and pressing on a submit

Ok, I understand, but are you saying you want to manually map each disqus_thread_id to an existing nodebb_topic_id?
just do it in an excel sheet, and export it as another CSV file, then the importer-plugin can use that, I am not gonna build an interface to enter these data, use excel - unless you think this can be done automatically, then I will help you with that

sanatisharif

@bentael

I don't get this idea of importing a CSV file. I mean the main point in my idea is that the plugin is going to fetch all of the threads out of the Disqus export for me and then I enter the correspondent NodeBB topic for each one .In this new idea of yours how is this gonna work?

sanatisharif

@bentael
I've been thinking about all of this and I think perhaps I get you now. Is your plugin going to do the insertion part for us? I mean are you saying that we give you that table I talked about (as a CSV or something) and then the plugin is going to add the Disqus posts according to that? In the other words this plugin is not going to fetch any Disqus posts from anythning!

bentael

@sanatisharif
actually you don't even need the CSV. All you have to do is:

export the XML file from Disqus
open that file in an editor, then "manually" add an field to each <thread> xml node, for example, i just added nbb:id="123456789" right next to the dsq:id

<thread dsq:id="2940898899" nbb:id="123456789">
    <id />
    <forum>sanatisharif</forum>
         <category dsq:id="2822236" />
    <link>http://sanatisharif.ir/Sanati-Sharif-Lesson/11/4/</link>
    <title>جزوات درس فیزیک</title>
    <message />
    <createdAt>2014-08-19T16:13:31Z</createdAt>
    <author>
    <email>[email protected]</email>
         <name>دبیرستان غیردولتی پسرانه دانشگ</name>
         <isAnonymous>false</isAnonymous>
         <username>sanatisharif</username>
    </author>
    <ipAddress>199.30.24.167</ipAddress>
    <isClosed>false</isClosed>
    <isDeleted>false</isDeleted>
</thread>

sanatisharif

@bentael

Hi again, We have built the XML file and here is the link to file :
http://dl1.sanatisharif.ir/disqus_remove_with_comment_tid.xml

Thanks for every thing.

bentael

@sanatisharif few questions.

ok, seems like all you want to do is import the disqus comments as NodeBB reply-posts to the relevant topics, minding the timestamps of the comments.
How come I don't see disqus enabled on your site? did you disable it?
some of the disqus threads are using the same nbb:id, is that correct? for example: dsq:id="2278882614" and dsq:id="2278946707" they are both using nbb:tid="1356"
I am gonna need a snapshot of your NodeBB dump to test against, dont share it publicly here as it may contain sensitive info, email it to [email protected]

This seems like one-time-use script, I can do it, but it's custom, it's not like a feature I can easily add to the -import plugin. The other thing is that I am gonna be every busy in the next 2 weeks, so that's gonna have to wait till early February

sanatisharif

@bentael

Thanks a lot . We appreciate your efforts so much.
Here are the answers to your questions:

Yes, that's exactly the case.
As you know we have migrated to NodeBB and we don't use Disqus anymore. Do you need Disqus to be enabled ? Why?
Yes , that's true . we have some Disqus threads that goes to the same NodeBB topic.
With all due respect we can't share this in any way. But an idea for doing this could be creating some dump posts in a local NodeBB and adding some Disqus threads to those posts to see whether the plugin works or not.

Finally about this being a one-time-script, I have to say that I disagree. I think this could be a plugin that helps anyone who wants to migrate from Disqus to NodeBB. Because in any case you have some Disqus threads that must go to a particular NodeBB topic. In the other words they gotta export their Disqus threads (which includes pots as well) and decide which thread goes to which NodeBB topic. Then they will be looking for a plugin to do the task of adding posts to NodeBB. Is that right?

sanatisharif

@bentael

we can add some input box to admin page of plugin, for customization blocks!

function setTopicId(json,thread,callback){
  /**
  * some customization.
  *
  **/
  callback(json,thread,tid);
}
function getNodebbTopicId(link){
 var id = -1 ;
 /**
 *each website has their own implementation.
 *
 **/
 return id;
}
function getCommentID(json, callback){

        var threads =  json.disqus.thread ;
        var itemsProcessed = 0;

        for (i = 0; i < threads.length; i++) {
            try{

                var link = parseInt(threads[i].link) ;
                var id = getNodebbTopicId(link);
                threads[i].$['nbb:tid'] = "";
                setTopicId(json,id,threads[i],function(json,thread,tid){
                    itemsProcessed++;
                    thread.$['nbb:tid'] = tid+"";
                    if(itemsProcessed === threads.length) {
                        callback(json);
                    }
                });
            }catch(ex){
                itemsProcessed++;
                console.error(ex);
            }

        }
};
var json = // Convert disqus xml file to json Object using some library, for example xml2js!

getCommentID(json, function(){
});

bentael

@sanatisharif said:

Do you need Disqus to be enabled ? Why?

No, i just wanted to able to verify the correctness by matching them manually.

With all due respect we can't share this in any way. But an idea for doing this could be creating some dump posts in a local NodeBB and adding some Disqus threads to those posts to see whether the plugin works or not.

That would work i guess.

helps anyone who wants to migrate from Disqus to NodeBB.

Not everyone has a nodebb-topic matching each disqus-thread, most people just use disqus as a comments system on a blogs, and migrating blogs to NodeBB isn't easy. The other issue that it involves manually adding that nbb;id thing, you had to do it manually.

Anyways, like i mentioned, I am out of town of the next 2 weeks, i'll look at this when I get back, feel free to remind me please.

sanatisharif

@bentael

The other issue that it involves manually adding that nbb;id thing, you had to do it manually.

We have implemented a piece of code for that and we were going to publish it . But even if he couldn't use the code , at least there is a plugin that does half of the way.

We can't thank you enough for being so much supportive and patient. See you in couple of weeks . Good luck.

sanatisharif

@bentael

Hi I'm back again , I hope you are fine and in the mode for doing this. So how do you feel? I say let's get this done .

FDX

I recently migrated to NodeBB from Disqus as well, it would be great to see some easy solution for this

By the way, @sanatisharif , since you are in the same situation you may be interested in reply/quote feature for nodebb-plugin-blog-comments. You have an example here (I think I need to change the font-size of that page ).

julian

@FDX Wow, looks great! Always nice to see it in the wild

sanatisharif

@FDX Thanks ,we are using this feature . It sounds like dear @bentael is in the house! is not in the house! I'm happy to see that I am not alone in this. I hope we will figure it out.

DaBoss3483

This post is deleted!

bentael

@sanatisharif i'm still around - let me see what i can do next weekend.

sanatisharif

@bentael
Thank a lot ,I'm looking forward to it.

sanatisharif

@bentael @julian
Hi there. I hope your are fine. Since there has not been a response from you for quite a while, I got worried . Are you OK? Is there a problem? I mean I feel like suddenly something bothered you and you just dumped us! We have been waiting for you.