Popular forum exporters
-
@lukelarris Thank you!
-
@psychobunny Thanks, some progress, new error though
[export-phpbb][debug] init()
[export-phpbb][info] Storage directory is: /home/a_5mith/Desktop/storage/home/a_5mith/nodebb/node_modules/nodebb-import-phpbb/lib/export.js:91
storage.initSync({dir: this.config.storageDir});
^
TypeError: Object #<Object> has no method 'initSync'
at Object.Export.init (/home/a_5mith/nodebb/node_modules/nodebb-import-phpbb/lib/export.js:91:11)
at Object.Export (/home/a_5mith/nodebb/node_modules/nodebb-import-phpbb/lib/export.js:62:8)
at Object.<anonymous> (/home/a_5mith/nodebb/node_modules/nodebb-import-phpbb/bin/export.js:37:2)
at Module._compile (module.js:456:26)
at Object.Module._extensions..js (module.js:474:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Function.Module.runMain (module.js:497:10)
at startup (node.js:119:16)
at node.js:902:3
Could be worth splitting these errors into individual topics for each forum software.
-
I'll try running this on a fresh install and see if I run into anything
-
@psychobunny Thanks buddy.
-
hey @a_5mith sorry for the delay! I found the error, looks like I was pulling the wrong repo for one of the packages. pull and npm install and I think you should be good to go
-
@psychobunny Hey buddy, thanks, installed without a problem, having a few issues getting it to connect to an external mysql connection, even with remote mysql configured. Keeps timing out on me. Will battle on and report my discoveries.
-
So, anyone got any experience with this?
Error: connect ETIMEDOUT
at errnoException (net.js:904:11)
at Object.afterConnect [as oncomplete] (net.js:895:19)Seems like it's just not connecting to the db (remote), so I added my Digital Ocean IP to my Remote MySQL page on my old host, this made no difference. I'm only doing a test run on a week old database backup, so I've got time to iron out these sorts of errors so I can minimise downtime when I take the leap.
-
Can't you set up a local copy of your DB? There isn't any throttling in the queries (ex. select 100 rows at a time), it basically does a select * per table, so if you have a really huge dataset I could see it timing out.
If not a possibility, I think you could try modifying the importer to send these commands initially to increase the timeout
-
The database is about 32mb (contains both an SMF database & Wordpress), 400 users, 25k posts, but times out within about 30 seconds of Exporting Users. Could probably do it locally thinking about it, will have a go and see what happens.
-
BTW after you finished with the phpBB export: I found that there's a bit of a bottleneck with regards to the redirect stuff in the importer, so if you don't care about writing redirects for SEO purposes then you can omit that and speed it up
-
I was going to bring up woltlab bb (I guess it's only popular in Germany), but actually first porting to phpbb sounds like a smart plan
-
Hey @psychobunny, took your advice and got mysql installed on Digital Ocean and imported it all that way, having a few issues with the final step of using the importer to actually get the data in. Following the steps, I get this error:
error: NodeBB automated setup didn't go too well.I already have nodebb setup, I noticed that import.config.json has a setup script built in, do I have to either a) remove it, or b) fill it out with the contents of nodebbs config.json?
-
Oh awesome, congrats for getting that far! I think at this point you'll have to ask the creator of the importer, @bentael - I didn't run into that error myself. If the actual import worked but creating the config didn't, yeah maybe you could create one yourself and see what happens?
-
@psychobunny Think I got it, removing
--flushand it's now going through users.Once I'm sure I know the process, I'll create a guide on the wiki page for anyone else that needs it.
-
awesome. hope it works out, I have no idea how long its going the import process is for something as big as yours
-
@psychobunny Users went over within about 5-10 minutes, however it's only transferred 5 of my 25,000 posts. So not really sure why that is.
5 threads, and only the OP, no replies.
-
yeah I found that posts take forever, about 500 posts took 3 minutes... so a rough extrapolation would suggest it would take you two and a half hours
Not sure if there are any optimizations that could be made in the importer, I imagine that @bentael has some ideas in mind for the future. For your sake I really hope it works the first try
(he has some "time machine" feature in there that lets you import just the first 100 posts or something, which I really recommend you trying to test drive everything) -
@psychobunny The posts are there, but got an error:
Importing Topics ...
[import][warn] [c:1] skipping topic:_tid:"2" --> _cid:valid: true _uid:valid: false
[import][warn] [c:2] skipping topic:_tid:"3" --> _cid:valid: true _uid:valid: false
[import][debug] [c:3] saving topic:_tid: 4
[import][debug] [c:4] saving topic:_tid: 6
[import][debug] [c:5] saving topic:_tid: 7
[import][debug] [c:6] saving topic:_tid: 8
[import][warn] [c:7] skipping topic:_tid:"9" --> _cid:valid: true _uid:valid: false
[import][debug] [c:8] saving topic:_tid: 10/home/a_5mith/nodebb/src/socket.io/index.js:199
var clients = io.sockets.clients();
^
TypeError: Cannot read property 'sockets' of undefinedTime taken I can live with, my forum is still running, this is a backup from last week, so have the time to check these things.
All the topics are there, it just seems that after 3 warnings it stops. I've even looked through 2 different topics one with and one without the warning, they look the same to me from a quick inspection.
-
TypeError: Cannot read property 'sockets' of undefinedhmm... my first guess is that your version of NodeBB isn't compatible with the importer. I know that it's good for 0.4.1 but not sure if anything new has broken it. Try downgrading to 0.4.1 (after the import, the upgrade command should take care of the rest)
What concerns me is the fact that an hour later you're still at tid 10
wonder what the bottleneck is in this process?
