Migrating to NodeBB - good stuff

  • @bentael I've heard discussions that we'd like to go live in the next 1-2 weeks and will want some time to confirm I have a stable data migration pipeline.

    Not sure what the timeline on 0.3.1 is...?

    I guess I should be okay to run edge for now until it get's there, though...

  • Plugin & Theme Dev

    @medwards oh - ok. Looks like 0.3.1 is not due till like a month.

    tell you what, I'll patch up to 0.3.0 so you can import to that, then it's just a matter of ./nodebb upgrade from there.

  • GNU/Linux Admin

    @bentael @medwards For the most part, we're limiting refactors to minor version bumps, and as the next version is 0.3.1, you should be fine if @bentael fixes up the importer for edge, even if 0.3.1 isn't released by then.

  • Plugin & Theme Dev

    @julian got you, i actually prefer doing that than falling back to 0.3.0

  • Plugin & Theme Dev

    Sorry a long ass reply with snapshots, it's 5:30am don't judge me.


    I think nodebb-plugin-import is ready for your set of eyes looking at it, if you can test it out with your real data (didn't publish to npm yet, still running some tests before I do that)

    here's what I did:

    • Basically made it compatible with 0.3.x (latest commit was 46e29dfb2b841ad4919059886b8c68f1c21da77e), however, and unfortunately, you still need to comment-out/change few lines of code in NodeBB's src, please read this readme section very carefully and click on each image to view the lines numbers to comment. Keep in mind that these line numbers are based on the commit-snapshot i mentioned in the beginning of this long sentence.
    • added both html-to-md and bbcode-to-md support to config.convert so you can choose either, and the performance is not that bad 8minutes (mac mini, i5, 16gb) to import that sample dump you sent me, look at the config section in the readme.
    • fixed locked issue, now ALL the topics are assumed unlocked with a flag, but then I relock them after all the posts get imported in
    • fixed pinned/unpinned issue
    • guarded against some the potential Errors you mentioned.

    give it a try, let me know, don't forget to revert the NodeBB src changes right after.
    Thanks for sending these sample dump files btw, nice work on the automated generation. I couldn't test the bbcode conversion since the dump content were all clear text, but I stole a function from here, then modified a little, see code here, I'll let you test it out.

    Also, I am not sure what's the difference between bb Code and vB code, can anyone explain?


    locking works with a reply posted to it 😄

    Screen Shot 2014-01-25 at 5.28.22 AM.png

    pinned threads now float to the top

    Screen Shot 2014-01-25 at 5.22.28 AM.png

    general migrated data looked good

    Screen Shot 2014-01-25 at 5.22.42 AM.png

  • GNU/Linux Admin

    @bentael Thanks so much for the work you're putting into this. You are providing an invaluable service to the community 🙂

  • Plugin & Theme Dev

    @julian de nada.
    hey do you the difference between vB Code and bb code ?

  • Admin

    wow, awesome @bentael

    vB code


    ubb code

    imo the one you used for ubb should be more than satisfactory for vB sites (just need to change the : to =)

  • Plugin & Theme Dev

    @psychobunny actually your second link should be http://en.wikipedia.org/wiki/BBCode

    UBB used straight HTML, at least that's what mine did.

    Anyways, looking at the bbcode vs the vB code, they look the same, to me al least. so the bbcode-to-html conversion should work fine.

  • @bentael Followed your instructions including code modifications - worked great! Pinned and locked threads are coming through as expected now, and the bbcode conversion sort of worked.

    The tags that are included in that regex list were properly replaced - I think I'm going to have to look through that link @psychobunny provided to ubbcode and add a few extra tags: [quote], [center] and maybe a few others, but some of the formatting came across nicely.

    Thanks for all the great work! I'll circle back once I get a chance to shore up the code converter to provide my updates. Your work got me to a nice stopping point for now and I'm going to have to prioritize our authentication/user creation stuff so I'll hopefully get back to the bbcode conversion a bit later in the week.

  • Plugin & Theme Dev

    @medwards great to hear ! feel free to edit that RegExp, I accept pull requests 😄 - and if it's a new format, we can add that too.

    Don't forget to check with your superiors to open-source that "vB exporter" when you have a chance. (name it something like nodebb-plugin-import-vbulletin to keep the convention) - if you don't have the time to write a full read-me and docs, that's fine, I can help out on that.

  • @bentael quick update: first draft of the vbexporter has been submitted for open-sourcing approval.

    It needs a bit more cleanup for general consumption ( it's my first node_module "release" ) but it's a decent starting point.

    Also I'm going to go ahead and fork your importer. Once I've got anything worth contributing back I'll figure out how to issue a pull request ( first time for that, too ).

    Thanks for all your work (and the node team too)! I'm excited to be able to contribute something back.

  • Plugin & Theme Dev

    @medwards sweet! once it's opened, I will fork it and give you a hand polishing it up.

  • @bentael @psychobunny @julian

    So I have approval to open source the vb exporter. It's rough at the moment and I won't have time to improve it a for a few weeks, but at least it'll be out there. I'll post a link later today.

    Quick question, also. After the most recent run of the importer my "admin" user (uid:1) does not appear to be an admin user. He's in the gid:2:members set which is the administrators group. I found an old post that said to add the id to the "administrators" global key in redis, but that doesn't appear to exist anymore.

    Any pointers?

  • Plugin & Theme Dev

    @medwards i may have an idea why, investigating..

  • Plugin & Theme Dev

    https://github.com/akhoury/nodebb-plugin-import/blob/master/lib/import.js#L522 that's a left over from my early nib code.

    but that may require a little more work to fix that i thought. If you already migrated, try to make the use an Admin manually.
    go to the /admin/users/search route, find that user, click on Make Admin.

  • Plugin & Theme Dev

    oh wait, im dumb, if do not have any admin users, you can't go the to /admin page.

    @medwards it's 'group:gid' global object,

    redis-cli hgetall group:gid
    # you should get an 'adminisitrator' field with value, most likely 2
    # then, assuming gid is 2, and the user.id that you want to make admin is also 2 (just a coincidence), so really the command is sadd gid:{gid}:members {uid}
    sadd gid:2:members 2

  • @bentael I think the database is already in the state you are trying to get me to put it in:

    redis> hgetall group:gid

    1. "registered-users"
    2. "1"
    3. "administrators"
    4. "2"

    redis> smembers gid:2:members

    1. "1"

    redis> hget user:1 username


    The user "admin" (uid:1) is already in the gid:2:members set (which is the "administrators" group).

    Are we sure the latest version didn't change the way privileges are allocated?

  • Plugin & Theme Dev

    @medwards your data looks correct, so you have 1 admin only, If you use the --flush flag with import, it WILL create the first admin user as admin:password because it is required by the NodeBB setup, basically just calling

    node app --setup="{ escaped JSON goes here}"

    to make sure you have the right DB schema, here are the default values. https://github.com/akhoury/nodebb-plugin-import/blob/master/lib/import.js#L79

    no matter if you have admins or in the imported DATA, the import WILL create the first admin for you, and tries to make YOUR imported-admins admins,

  • Plugin & Theme Dev

    that's why your first user with id 1 has id 2, so .. all ids are incremented by one.
    One way to avoid this is to set your main admin as the default Admin in the config.setupVal values, and delete its _uid.1 file. this way, it will be the one to be imported as the main admin, and all of its posts will be owned by that same one.

Suggested Topics

| | |

© 2014 – 2022 NodeBB, Inc. — Made in Canada.