NodeBB to IPS migration tool

General Discussion
  • Hi there!

    First of all, I hope I won't get banned right now 🙂 Personally I do like NodeBB - well done, guys! I've been using it since 0.4.x and I submit bug reports/PRs from time to time (unfortunately, I'm not familiar with NodeJS well. Anyway, I'm still using it for one of my communities. At the same time, some communities do prefer classic forum solutions which look like it was 10 years ago and there is nothing we can do. One of my forums has been running NodeBB for 1 year and my visitors asks me everyday to patch something. I had to find some solution to move to any other 'classic' platform. There were no any ready-to-use migration tools so I decided to code a bit 🙂

    So, here it is:

    A few words about it:

    • A tool for NodeBB to IPS 4.x migration
    • Written in pure Java (yeah, I'm waiting for your blaming comments 🙂 )
    • Works with NodeBB 0.7.x (tested with 0.7.0, 0.7.2) Redis-based, IPS 4.x (tested with 4.0.11) MySQL
    • It does not uses any forums APIs and works only with DBs

    Currently the following content migration supported:

    • Users
    • Groups
    • Categories (forums), subcategories
    • Topics
    • Posts, post formatting
    • Internal cross-links (links to other posts/topics)
    • User references
    • Private Messages (chats)
    • Reputation
    • Users Followers/Following
    • Topics Followers

    What is not supported:

    • Passwords (it seems there is no way to do that, but I haven't looked into IPS code yet)
    • Unread topic markers (possible, but a bit complicated due to extremely weird IPS unread posts markers storage format)
    • Access rights (users/groups ones) (possible too, but it's easier to configure on your own as IPS provides a bit more options)
    • Groups title images (had no time, sorry 🙂 )
    • Tags (same)

    It implements ID mapping so all the content may be imported to live IPS with some content there (not tested yet, but it should work fine, the only thing to check is username collisions).

    Another extra features are links parsing (all the internal cross-links will be reformatted), user references, last activities for users (IPS use an extra table for that, huh), unread private messages markers etc. Profile avatars migrated as-is, IPS does not allow to use external URLs (I could not find such option, but may be its hidden somewhere), so all external avatars downloaded to a local storage too. All the posts content is processed with Markdown processor (PegDown, great thing btw), so posts formatting is ok.

    I should say that it was not so easy as it was supposed to be. IPS is weird a bit. For example, I had to look into the sources to understand how keys are generated in their tables - it uses something like md5(value1 + value2) and value1, value2 stored in the same table, it looks strange 🙂 Anyway, it works fine now. And I'm feeling like a PHP guru 😛

    Actually, it can be adopted to any other forum solution as it loads all the NodeBB data into memory and normalize it, so there should not be any problems with that.

    Here is a link to the russian post :

    P.S. It's not optimized at all, keep it in mind if you want to use it. It does not use batches etc. The reason is quite simple - I have only 8k members community so it's not a 100k one and it was much easier to debug the solution implemented in such way. Anyway, if someone is interested, I can improve it a bit. At the same time its ok for my personal needs.

    I'll be glad to answer your questions 😉

    Best wishes!

  • Updated to support IPS latest patch (some groups and members changes). Tested again with 0.7.2 NodeBB and IPS.

    In fact, there is a schema validation built upon Hibernate JPA, so if something is wrong with DB - it will fail at startup with no data corruption.

Suggested Topics