Popular forum exporters
-
@psychobunny The first time I cloned it from your github page as I didn't notice it was on npmjs, so I've removed it, and run
npm install nodebb-plugin-import-phpbbwhich gave me the following errors
npm ERR! git clone git://github.com/psychobunny/node-persist.git Cloning into bare repository '/home/a_5mith/.npm/_git-remotes/git-github-com-psychobunny-node-persist-git-39cb9d69'...
npm ERR! git clone git://github.com/psychobunny/node-persist.git
npm ERR! git clone git://github.com/psychobunny/node-persist.git fatal: remote error:
npm ERR! git clone git://github.com/psychobunny/node-persist.git Repository not found.Not quite sure what's going on.
-
No idea about that error hmm, you might just be better off cloning and npm installing from there (that's how I got it working on my end)
-
@lukelarris Thank you!
-
@psychobunny Thanks, some progress, new error though
[export-phpbb][debug] init()
[export-phpbb][info] Storage directory is: /home/a_5mith/Desktop/storage/home/a_5mith/nodebb/node_modules/nodebb-import-phpbb/lib/export.js:91
storage.initSync({dir: this.config.storageDir});
^
TypeError: Object #<Object> has no method 'initSync'
at Object.Export.init (/home/a_5mith/nodebb/node_modules/nodebb-import-phpbb/lib/export.js:91:11)
at Object.Export (/home/a_5mith/nodebb/node_modules/nodebb-import-phpbb/lib/export.js:62:8)
at Object.<anonymous> (/home/a_5mith/nodebb/node_modules/nodebb-import-phpbb/bin/export.js:37:2)
at Module._compile (module.js:456:26)
at Object.Module._extensions..js (module.js:474:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Function.Module.runMain (module.js:497:10)
at startup (node.js:119:16)
at node.js:902:3
Could be worth splitting these errors into individual topics for each forum software.
-
I'll try running this on a fresh install and see if I run into anything
-
@psychobunny Thanks buddy.
-
hey @a_5mith sorry for the delay! I found the error, looks like I was pulling the wrong repo for one of the packages. pull and npm install and I think you should be good to go
-
@psychobunny Hey buddy, thanks, installed without a problem, having a few issues getting it to connect to an external mysql connection, even with remote mysql configured. Keeps timing out on me. Will battle on and report my discoveries.
-
So, anyone got any experience with this?
Error: connect ETIMEDOUT
at errnoException (net.js:904:11)
at Object.afterConnect [as oncomplete] (net.js:895:19)Seems like it's just not connecting to the db (remote), so I added my Digital Ocean IP to my Remote MySQL page on my old host, this made no difference. I'm only doing a test run on a week old database backup, so I've got time to iron out these sorts of errors so I can minimise downtime when I take the leap.
-
Can't you set up a local copy of your DB? There isn't any throttling in the queries (ex. select 100 rows at a time), it basically does a select * per table, so if you have a really huge dataset I could see it timing out.
If not a possibility, I think you could try modifying the importer to send these commands initially to increase the timeout
-
The database is about 32mb (contains both an SMF database & Wordpress), 400 users, 25k posts, but times out within about 30 seconds of Exporting Users. Could probably do it locally thinking about it, will have a go and see what happens.
-
BTW after you finished with the phpBB export: I found that there's a bit of a bottleneck with regards to the redirect stuff in the importer, so if you don't care about writing redirects for SEO purposes then you can omit that and speed it up
-
I was going to bring up woltlab bb (I guess it's only popular in Germany), but actually first porting to phpbb sounds like a smart plan
-
Hey @psychobunny, took your advice and got mysql installed on Digital Ocean and imported it all that way, having a few issues with the final step of using the importer to actually get the data in. Following the steps, I get this error:
error: NodeBB automated setup didn't go too well.I already have nodebb setup, I noticed that import.config.json has a setup script built in, do I have to either a) remove it, or b) fill it out with the contents of nodebbs config.json?
-
Oh awesome, congrats for getting that far! I think at this point you'll have to ask the creator of the importer, @bentael - I didn't run into that error myself. If the actual import worked but creating the config didn't, yeah maybe you could create one yourself and see what happens?
-
@psychobunny Think I got it, removing
--flushand it's now going through users.Once I'm sure I know the process, I'll create a guide on the wiki page for anyone else that needs it.
-
awesome. hope it works out, I have no idea how long its going the import process is for something as big as yours
-
@psychobunny Users went over within about 5-10 minutes, however it's only transferred 5 of my 25,000 posts. So not really sure why that is.
5 threads, and only the OP, no replies.
-
yeah I found that posts take forever, about 500 posts took 3 minutes... so a rough extrapolation would suggest it would take you two and a half hours
Not sure if there are any optimizations that could be made in the importer, I imagine that @bentael has some ideas in mind for the future. For your sake I really hope it works the first try
(he has some "time machine" feature in there that lets you import just the first 100 posts or something, which I really recommend you trying to test drive everything)
