[nodebb-plugin-import] Import your forum to NBB


  • Community Rep

    @alff0x1f said in [nodebb-plugin-import] Import your forum to NBB:

    I don't know why it is not support for actual versions

    It's because it takes a lot of work to keep the plugin up to date with the newest version. We know it works for v1.0.0 but since it touches some internal api's that aren't necessarily supposed to be developer-facing, it will encounter more breakages than say, a normal plugin.


  • Plugin & Theme Dev

    I will be upping support soon to a more recent version. Sorry guys it's been a few crazy months and didn't get a chance to work on it.



  • Great to hear - looking forward to it!



  • @bentael I'm got a new forum set up with v1.0.0 and everything seems to work, except I get an "invalid csrf token" error when I try to log in. I have nodebb listening on a public facing interface, no proxying. The URL I'm using to access the site matches the one in config.json, it's just the server IP. I had no problems earlier with a different db and the most recent nodebb release, but the importer obviously won't work there.

    Thanks for any help.


  • Global Moderator

    @GaryBusey make sure sockets are working. Check for any errors on the client or server. Try running nodebb in dev mode with ./nodebb dev.



  • Has anyone managed to import a fairly large mybb forum with the importer ? I wanted to migrate from an old mybb1.6 forum with ~5000threads, 15000 members and around 1.5M individual posts. I first migrated that forum to mybb1.8 then installed nodebb 1.0.0 as per the nodebb-plugin-import github readme.

    The import works for a while but nodebb eventually crash before all the posts are imported.

    I tried cleaning up the original database with only sticky threads + threads whose latest post was less than 2y old. Still didn't manage to do a successfull import. Sadly resuming the import creates duplicated threads.

    One physical machine with 16GB of ram, original database in a mariadb docker container, destination DB on mongodb 3.4.1 running in the official docker container, nodebb on another docker container based on the official ubuntu xenial image.

    Thanks
    Thomas


  • Plugin & Theme Dev

    @oPignonLibre there is a memory leak, i've known of this issue, but I am refactoring the importer plugin.

    here's what you can do if the crash is because you ran out of memory,

    1. go back to NodeBB Admin panel > Plugins > Import
    2. Resume the import by clicking on "Last import was interrupted, try to resume"
    3. the import will start to re-import all the posts, but it will skip the ones already imported

    With 1.5M posts, you might have to do this a couple of times - I have definitely imported over 2M posts, and once 60M, it wasn't fun, but doable.

    I just moved homes and I am still settling in, I will wrap up my refactor soon, and it will support NodeBB 1.4+ too, sorry the inconvenience.


  • Plugin & Theme Dev

    wait, sorry I just read that.

    Sadly resuming the import creates duplicated threads.

    really? I have never had this issue. I will take a look, can you give me a small sample of data, and link me to the mybb exporter that you're using?


  • Plugin & Theme Dev

    send it to bentael@gmail.com



  • @bentael yep. I import from phpbb3 last september and remember about duplication.

    @oPignonLibre try create big swap before you import


  • Plugin & Theme Dev

    @alff0x1f Humm.. i will take a look



  • Sorry for the late reply,

    I will look at creating some additionnal swap space before importation. I didn't monitor memory usage during initial attempts.

    I will try first with a smaller sample and will send it to you if duplication still happens. I used asciicat's mybb exporter.

    Thanks



  • Hi,

    Trying to port a Kunena forum to Nodebb (veaf.org/fr/forums -> community.veaf.org).

    I tried https://github.com/eyesofkids/nodebb-plugin-import-kunena .

    Looks not too far from the goal 😊 .

    Users and categories import looks good.
    Just a lot of topics and messages missing...

    The log (minus repetitive errors for topics and messages import) :
    logs.txt

    Maybe the problem is with topics import and the related error error:user-too-new ?

    Possible to help me solving this issue ? That would consolidate another importer for NodeBB 😊 .

    Regards.



  • Hiho,

    is this plugin still maintained? I am asking because we would like to migrate our board with ~25M posts. And the supported version 1.0.3 is nearly two years old. So we would like to do a migration over the course of a month or so, while we keep syncing new/changed content to the new forum, while the old is still in place.

    It would not be very practical to move once, and then upgrade, as the forum would have to be down for quite some time, while the migration process is running for a long time. Not even able to change configuration before going live.

    Thanks for replying.



  • 2018-04-16T18:11:24.857Z [5024] - error: TypeError: path must be a string or Buffer
    at Object.fs.readdirSync (fs.js:904:18)
    at getSiblingDirectories (C:\Users\root\nodebb\node_modules\nodebb-plugin-import\node_modules\nodebb-plugin-require\index.js:48:19)
    at findPackageDirectory (C:\Users\root\nodebb\node_modules\nodebb-plugin-import\node_modules\nodebb-plugin-require\index.js:25:20)
    at findNodebbDirectory (C:\Users\root\nodebb\node_modules\nodebb-plugin-import\node_modules\nodebb-plugin-require\index.js:59:9)
    at Object.<anonymous> (C:\Users\root\nodebb\node_modules\nodebb-plugin-import\node_modules\nodebb-plugin-require\index.js:62:16)
    at Module._compile (module.js:652:30)
    at Object.Module._extensions..js (module.js:663:10)
    at Module.load (module.js:565:32)
    at tryModuleLoad (module.js:505:12)
    at Function.Module._load (module.js:497:3)
    at Module.require (module.js:596:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (C:\Users\root\nodebb\node_modules\nodebb-plugin-import\server\index.js:1:80)
    at Module._compile (module.js:652:30)
    at Object.Module._extensions..js (module.js:663:10)
    at Module.load (module.js:565:32)
    2018-04-16T18:11:24.857Z [5024] - warn: [plugins] Unable to parse library for: nodebb-plugin-import


  • Gamers

    Any plans on updating this for the new version of IPB? I've currently got IPB 4.2.6 installed, which I've used for the last 14 years with 1.7 million posts and 32k members.

    Would like to migrate the data, if possible.



  • This plugin supports nBB v1.12.1 according to GitHub

    I tried to install this today via ACP and then terminal, but both times I got this funky error. I have no clue what is going on here.

    gyp ERR! build error 
    gyp ERR! stack Error: not found: make
    gyp ERR! stack     at getNotFoundError (/usr/lib/node_modules/npm/node_modules/which/which.js:13:12)
    gyp ERR! stack     at F (/usr/lib/node_modules/npm/node_modules/which/which.js:68:19)
    gyp ERR! stack     at E (/usr/lib/node_modules/npm/node_modules/which/which.js:80:29)
    gyp ERR! stack     at /usr/lib/node_modules/npm/node_modules/which/which.js:89:16
    gyp ERR! stack     at /usr/lib/node_modules/npm/node_modules/isexe/index.js:42:5
    gyp ERR! stack     at /usr/lib/node_modules/npm/node_modules/isexe/mode.js:8:5
    gyp ERR! stack     at FSReqCallback.oncomplete (fs.js:158:21)
    gyp ERR! System Linux 4.15.0-47-generic
    gyp ERR! command "/usr/bin/node" "/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
    gyp ERR! cwd /home/node/nodebb/node_modules/iconv
    gyp ERR! node -v v11.15.0
    gyp ERR! node-gyp -v v3.8.0
    gyp ERR! not ok 
    npm WARN textcomplete.contenteditable@0.1.1 requires a peer of textcomplete@^0.14.2 but none is installed. You must install peer dependencies yourself.
    
    npm ERR! code ELIFECYCLE
    npm ERR! errno 1
    npm ERR! iconv@2.3.4 install: `node-gyp rebuild`
    npm ERR! Exit status 1
    npm ERR! 
    npm ERR! Failed at the iconv@2.3.4 install script.
    npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
    
    npm ERR! A complete log of this run can be found in:
    npm ERR!     /home/node/.npm/_logs/2019-06-18T13_25_21_688Z-debug.log
    

    The debug log is massive I can post if women want to take a look.


  • Community Rep

    what OS are you running on?



  • @Bri ubuntu (18.04 I think.)



  • Try with nodejs 8/10/12


Log in to reply
 

Suggested Topics

  • 3
  • 1
  • 1
  • 171
  • 3
| |