[nodebb-plugin-import] Import your forum to NBB

[bentael]

@psychobunny I will thank you !

[Tom Wilson]

Apologies for resurrecting this topic, but is this plugin available for v1.4 of NodeBB? I don't see it listed in the list of available plugins on the admin panel.

[alff0x1f]

@Tom-Wilson Unfortunately

LATEST VERSION ONLY SUPPORTS:
NodeBB v1.0.0

via https://github.com/akhoury/nodebb-plugin-import

I install nodeBB v1.0.0 and update my forum to 1.2.0 after import information from my old forum (v 1.2.0 was actual when I import from phpbb3)
(IMHO it is most important plugin for commtity grow and I don't know why it is not support for actual versions)

[Tom Wilson]

Ah that's a shame. I just tried to downgrade, but encountered all sorts of problems :(.

[Bri]

@Tom-Wilson if you're importing an old forum, I'm assuming you'd be starting from scratch. Just reset your database and do git checkout v1.0.0 and then the regular setup steps from there.

If you're trying to merge with existing data already in NBB, that's another thing entirely.

[Bri]

@alff0x1f said in [nodebb-plugin-import] Import your forum to NBB:

I don't know why it is not support for actual versions

It's because it takes a lot of work to keep the plugin up to date with the newest version. We know it works for v1.0.0 but since it touches some internal api's that aren't necessarily supposed to be developer-facing, it will encounter more breakages than say, a normal plugin.

[bentael]

I will be upping support soon to a more recent version. Sorry guys it's been a few crazy months and didn't get a chance to work on it.

[Tom Wilson]

Great to hear - looking forward to it!

[GaryBusey]

@bentael I'm got a new forum set up with v1.0.0 and everything seems to work, except I get an "invalid csrf token" error when I try to log in. I have nodebb listening on a public facing interface, no proxying. The URL I'm using to access the site matches the one in config.json, it's just the server IP. I had no problems earlier with a different db and the most recent nodebb release, but the importer obviously won't work there.

Thanks for any help.

[PitaJ]

@GaryBusey make sure sockets are working. Check for any errors on the client or server. Try running nodebb in dev mode with ./nodebb dev.

[oPignonLibre]

Has anyone managed to import a fairly large mybb forum with the importer ? I wanted to migrate from an old mybb1.6 forum with ~5000threads, 15000 members and around 1.5M individual posts. I first migrated that forum to mybb1.8 then installed nodebb 1.0.0 as per the nodebb-plugin-import github readme.

The import works for a while but nodebb eventually crash before all the posts are imported.

I tried cleaning up the original database with only sticky threads + threads whose latest post was less than 2y old. Still didn't manage to do a successfull import. Sadly resuming the import creates duplicated threads.

One physical machine with 16GB of ram, original database in a mariadb docker container, destination DB on mongodb 3.4.1 running in the official docker container, nodebb on another docker container based on the official ubuntu xenial image.

Thanks
Thomas

[bentael]

@oPignonLibre there is a memory leak, i've known of this issue, but I am refactoring the importer plugin.

here's what you can do if the crash is because you ran out of memory,

1. go back to NodeBB Admin panel > Plugins > Import
2. Resume the import by clicking on "Last import was interrupted, try to resume"
3. the import will start to re-import all the posts, but it will skip the ones already imported

With 1.5M posts, you might have to do this a couple of times - I have definitely imported over 2M posts, and once 60M, it wasn't fun, but doable.

I just moved homes and I am still settling in, I will wrap up my refactor soon, and it will support NodeBB 1.4+ too, sorry the inconvenience.

[bentael]

wait, sorry I just read that.

Sadly resuming the import creates duplicated threads.

really? I have never had this issue. I will take a look, can you give me a small sample of data, and link me to the mybb exporter that you're using?

[bentael]

send it to [email protected]

[alff0x1f]

@bentael yep. I import from phpbb3 last september and remember about duplication.

@oPignonLibre try create big swap before you import

[bentael]

@alff0x1f Humm.. i will take a look

[oPignonLibre]

Sorry for the late reply,

I will look at creating some additionnal swap space before importation. I didn't monitor memory usage during initial attempts.

I will try first with a smaller sample and will send it to you if duplication still happens. I used asciicat's mybb exporter.

Thanks

[Couby]

Hi,

Trying to port a Kunena forum to Nodebb (veaf.org/fr/forums -> community.veaf.org).

I tried https://github.com/eyesofkids/nodebb-plugin-import-kunena .

Looks not too far from the goal .

Users and categories import looks good.
Just a lot of topics and messages missing...

The log (minus repetitive errors for topics and messages import) :
logs.txt

Maybe the problem is with topics import and the related error error:user-too-new ?

Possible to help me solving this issue ? That would consolidate another importer for NodeBB .

Regards.

[Arany]

Hiho,

is this plugin still maintained? I am asking because we would like to migrate our board with ~25M posts. And the supported version 1.0.3 is nearly two years old. So we would like to do a migration over the course of a month or so, while we keep syncing new/changed content to the new forum, while the old is still in place.

It would not be very practical to move once, and then upgrade, as the forum would have to be down for quite some time, while the migration process is running for a long time. Not even able to change configuration before going live.

Thanks for replying.

[Crushed Ice]

2018-04-16T18:11:24.857Z [5024] - error: TypeError: path must be a string or Buffer
at Object.fs.readdirSync (fs.js:904:18)
at getSiblingDirectories (C:\Users\root\nodebb\node_modules\nodebb-plugin-import\node_modules\nodebb-plugin-require\index.js:48:19)
at findPackageDirectory (C:\Users\root\nodebb\node_modules\nodebb-plugin-import\node_modules\nodebb-plugin-require\index.js:25:20)
at findNodebbDirectory (C:\Users\root\nodebb\node_modules\nodebb-plugin-import\node_modules\nodebb-plugin-require\index.js:59:9)
at Object.<anonymous> (C:\Users\root\nodebb\node_modules\nodebb-plugin-import\node_modules\nodebb-plugin-require\index.js:62:16)
at Module._compile (module.js:652:30)
at Object.Module._extensions..js (module.js:663:10)
at Module.load (module.js:565:32)
at tryModuleLoad (module.js:505:12)
at Function.Module._load (module.js:497:3)
at Module.require (module.js:596:17)
at require (internal/module.js:11:18)
at Object.<anonymous> (C:\Users\root\nodebb\node_modules\nodebb-plugin-import\server\index.js:1:80)
at Module._compile (module.js:652:30)
at Object.Module._extensions..js (module.js:663:10)
at Module.load (module.js:565:32)
2018-04-16T18:11:24.857Z [5024] - warn: [plugins] Unable to parse library for: nodebb-plugin-import