[nodebb-plugin-import] Import your forum to NBB
-
Great to hear - looking forward to it!
-
@bentael I'm got a new forum set up with v1.0.0 and everything seems to work, except I get an "invalid csrf token" error when I try to log in. I have nodebb listening on a public facing interface, no proxying. The URL I'm using to access the site matches the one in config.json, it's just the server IP. I had no problems earlier with a different db and the most recent nodebb release, but the importer obviously won't work there.
Thanks for any help.
-
@GaryBusey make sure sockets are working. Check for any errors on the client or server. Try running nodebb in dev mode with
./nodebb dev. -
Has anyone managed to import a fairly large mybb forum with the importer ? I wanted to migrate from an old mybb1.6 forum with ~5000threads, 15000 members and around 1.5M individual posts. I first migrated that forum to mybb1.8 then installed nodebb 1.0.0 as per the nodebb-plugin-import github readme.
The import works for a while but nodebb eventually crash before all the posts are imported.
I tried cleaning up the original database with only sticky threads + threads whose latest post was less than 2y old. Still didn't manage to do a successfull import. Sadly resuming the import creates duplicated threads.
One physical machine with 16GB of ram, original database in a mariadb docker container, destination DB on mongodb 3.4.1 running in the official docker container, nodebb on another docker container based on the official ubuntu xenial image.
Thanks
Thomas -
@oPignonLibre there is a memory leak, i've known of this issue, but I am refactoring the importer plugin.
here's what you can do if the crash is because you ran out of memory,
- go back to NodeBB Admin panel > Plugins > Import
- Resume the import by clicking on "Last import was interrupted, try to resume"
- the import will start to re-import all the posts, but it will skip the ones already imported
With 1.5M posts, you might have to do this a couple of times - I have definitely imported over 2M posts, and once 60M, it wasn't fun, but doable.
I just moved homes and I am still settling in, I will wrap up my refactor soon, and it will support NodeBB 1.4+ too, sorry the inconvenience.
-
wait, sorry I just read that.
Sadly resuming the import creates duplicated threads.
really? I have never had this issue. I will take a look, can you give me a small sample of data, and link me to the mybb exporter that you're using?
-
send it to [email protected]
-
Sorry for the late reply,
I will look at creating some additionnal swap space before importation. I didn't monitor memory usage during initial attempts.
I will try first with a smaller sample and will send it to you if duplication still happens. I used asciicat's mybb exporter.
Thanks
-
Hi,
Trying to port a Kunena forum to Nodebb (veaf.org/fr/forums -> community.veaf.org).
I tried https://github.com/eyesofkids/nodebb-plugin-import-kunena .
Looks not too far from the goal
.Users and categories import looks good.
Just a lot of topics and messages missing...The log (minus repetitive errors for topics and messages import) :
logs.txtMaybe the problem is with topics import and the related error error:user-too-new ?
Possible to help me solving this issue ? That would consolidate another importer for NodeBB
.Regards.
-
Hiho,
is this plugin still maintained? I am asking because we would like to migrate our board with ~25M posts. And the supported version 1.0.3 is nearly two years old. So we would like to do a migration over the course of a month or so, while we keep syncing new/changed content to the new forum, while the old is still in place.
It would not be very practical to move once, and then upgrade, as the forum would have to be down for quite some time, while the migration process is running for a long time. Not even able to change configuration before going live.
Thanks for replying.
-
2018-04-16T18:11:24.857Z [5024] - [31merror[39m: TypeError: path must be a string or Buffer
at Object.fs.readdirSync (fs.js:904:18)
at getSiblingDirectories (C:\Users\root\nodebb\node_modules\nodebb-plugin-import\node_modules\nodebb-plugin-require\index.js:48:19)
at findPackageDirectory (C:\Users\root\nodebb\node_modules\nodebb-plugin-import\node_modules\nodebb-plugin-require\index.js:25:20)
at findNodebbDirectory (C:\Users\root\nodebb\node_modules\nodebb-plugin-import\node_modules\nodebb-plugin-require\index.js:59:9)
at Object.<anonymous> (C:\Users\root\nodebb\node_modules\nodebb-plugin-import\node_modules\nodebb-plugin-require\index.js:62:16)
at Module._compile (module.js:652:30)
at Object.Module._extensions..js (module.js:663:10)
at Module.load (module.js:565:32)
at tryModuleLoad (module.js:505:12)
at Function.Module._load (module.js:497:3)
at Module.require (module.js:596:17)
at require (internal/module.js:11:18)
at Object.<anonymous> (C:\Users\root\nodebb\node_modules\nodebb-plugin-import\server\index.js:1:80)
at Module._compile (module.js:652:30)
at Object.Module._extensions..js (module.js:663:10)
at Module.load (module.js:565:32)
2018-04-16T18:11:24.857Z [5024] - [33mwarn[39m: [plugins] Unable to parse library for: nodebb-plugin-import -
This plugin supports nBB v1.12.1 according to GitHub
I tried to install this today via ACP and then terminal, but both times I got this funky error. I have no clue what is going on here.
gyp ERR! build error gyp ERR! stack Error: not found: make gyp ERR! stack at getNotFoundError (/usr/lib/node_modules/npm/node_modules/which/which.js:13:12) gyp ERR! stack at F (/usr/lib/node_modules/npm/node_modules/which/which.js:68:19) gyp ERR! stack at E (/usr/lib/node_modules/npm/node_modules/which/which.js:80:29) gyp ERR! stack at /usr/lib/node_modules/npm/node_modules/which/which.js:89:16 gyp ERR! stack at /usr/lib/node_modules/npm/node_modules/isexe/index.js:42:5 gyp ERR! stack at /usr/lib/node_modules/npm/node_modules/isexe/mode.js:8:5 gyp ERR! stack at FSReqCallback.oncomplete (fs.js:158:21) gyp ERR! System Linux 4.15.0-47-generic gyp ERR! command "/usr/bin/node" "/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild" gyp ERR! cwd /home/node/nodebb/node_modules/iconv gyp ERR! node -v v11.15.0 gyp ERR! node-gyp -v v3.8.0 gyp ERR! not ok npm WARN [email protected] requires a peer of textcomplete@^0.14.2 but none is installed. You must install peer dependencies yourself. npm ERR! code ELIFECYCLE npm ERR! errno 1 npm ERR! [email protected] install: `node-gyp rebuild` npm ERR! Exit status 1 npm ERR! npm ERR! Failed at the [email protected] install script. npm ERR! This is probably not a problem with npm. There is likely additional logging output above. npm ERR! A complete log of this run can be found in: npm ERR! /home/node/.npm/_logs/2019-06-18T13_25_21_688Z-debug.logThe debug log is massive I can post if women want to take a look.
-
@baris said in [nodebb-plugin-import] Import your forum to NBB:
Try with nodejs 8/10/12
Any point in running npm audit fix on current version of nodejs?
Meanwhile, let's do this - upgrade from node 11.15.0 npm 6.7.0 to node 12.
To note. I experienced (unresolved) erros in past when installing fresh nodejs 12 when it came to the nodeBB install (v1.12).
If that doesn't work I'll try a downgrade to nodejs 10!!!
-
errrggggghhh.... used n to upgrade/downgrade and with between versions of nodejs , ran the instal via terminal. same deal, now onto
Err msg node12
gyp ERR! build error gyp ERR! stack Error: not found: make gyp ERR! stack at getNotFoundError (/usr/local/lib/node_modules/npm/node_modules/which/which.js:13:12) gyp ERR! stack at F (/usr/local/lib/node_modules/npm/node_modules/which/which.js:68:19) gyp ERR! stack at E (/usr/local/lib/node_modules/npm/node_modules/which/which.js:80:29) gyp ERR! stack at /usr/local/lib/node_modules/npm/node_modules/which/which.js:89:16 gyp ERR! stack at /usr/local/lib/node_modules/npm/node_modules/isexe/index.js:42:5 gyp ERR! stack at /usr/local/lib/node_modules/npm/node_modules/isexe/mode.js:8:5 gyp ERR! stack at FSReqCallback.oncomplete (fs.js:149:21) gyp ERR! System Linux 4.15.0-51-generic gyp ERR! command "/usr/local/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild" gyp ERR! cwd /home/node/nodebb/node_modules/iconv gyp ERR! node -v v12.3.1 gyp ERR! node-gyp -v v3.8.0 gyp ERR! not ok npm WARN [email protected] requires a peer of textcomplete@^0.14.2 but none is installed. You must install peer dependencies yourself. npm ERR! code ELIFECYCLE npm ERR! errno 1 npm ERR! [email protected] install: `node-gyp rebuild` npm ERR! Exit status 1 npm ERR! npm ERR! Failed at the [email protected] install script. npm ERR! This is probably not a problem with npm. There is likely additional logging output above. npm ERR! A complete log of this run can be found in: npm ERR! /home/node/.npm/_logs/2019-06-18T20_28_54_773Z-debug.logErr msg node10
gyp ERR! build error gyp ERR! stack Error: not found: make gyp ERR! stack at getNotFoundError (/usr/local/lib/node_modules/npm/node_modules/which/which.js:13:12) gyp ERR! stack at F (/usr/local/lib/node_modules/npm/node_modules/which/which.js:68:19) gyp ERR! stack at E (/usr/local/lib/node_modules/npm/node_modules/which/which.js:80:29) gyp ERR! stack at /usr/local/lib/node_modules/npm/node_modules/which/which.js:89:16 gyp ERR! stack at /usr/local/lib/node_modules/npm/node_modules/isexe/index.js:42:5 gyp ERR! stack at /usr/local/lib/node_modules/npm/node_modules/isexe/mode.js:8:5 gyp ERR! stack at FSReqWrap.oncomplete (fs.js:153:21) gyp ERR! System Linux 4.15.0-51-generic gyp ERR! command "/usr/local/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild" gyp ERR! cwd /home/node/nodebb/node_modules/iconv gyp ERR! node -v v10.16.0 gyp ERR! node-gyp -v v3.8.0 gyp ERR! not ok npm WARN [email protected] requires a peer of textcomplete@^0.14.2 but none is installed. You must install peer dependencies yourself. npm ERR! code ELIFECYCLE npm ERR! errno 1 npm ERR! [email protected] install: `node-gyp rebuild` npm ERR! Exit status 1 npm ERR! npm ERR! Failed at the [email protected] install script. npm ERR! This is probably not a problem with npm. There is likely additional logging output above. npm ERR! A complete log of this run can be found in: npm ERR! /home/node/.npm/_logs/2019-06-18T20_32_32_662Z-debug.logAlso got a similar error for node8 too.
