Source forums exporters to NodeBB, I need your DB Dump.

[bentael]

@Steve that's cool, you can shut off the DigitalOcean node, I have a few running and I test on my home machine.

So tonight I was able to import your DB from 2 tries only, time took about 10min total.

but I had to:

• use the hashtalk branch
• set the redis temp config,
• make this manual change in NodeBB core, proposed PR

Anyways, I think it's better to wait for 'hashtalk' to merge into master then 0.5.1, i'll keep on testing meanwhile.
Thanks for being patient.

looks familiar?

[baris]

Hashtalk has been merged into master fyi.

[Steve]

@bentael said:

@Steve that's cool, you can shut off the DigitalOcean node, I have a few running and I test on my home machine.

So tonight I was able to import your DB from 2 tries only, time took about 10min total.

but I had to:

• use the hashtalk branch
• set the redis temp config,
• make this manual change in NodeBB core, proposed PR

Anyways, I think it's better to wait for 'hashtalk' to merge into master then 0.5.1, i'll keep on testing meanwhile.
Thanks for being patient.

looks familiar?

hahah wow nice job!! that's a fairly stock vB database too with the exception of the Garage which we didn't plan on using moving forward since it's more like a graveyard.. people post more recent/daily kind of pics on FB nowadays.

I'm excited to theme it out offered style. i've already started chopping up pics and whatnot to get it ready to rock.

[bentael]

jeezaloo!

dat NodeBB master branch is lightning fast!

@Steve with a combination of [email protected] and [email protected] I was able to import ur DB in less than 5min. Consistently.

[nodebb-plugin-import] importer.success Importing 632 users.
[nodebb-plugin-import] importer.success Importing 632/632 users took: 3.63 seconds
[nodebb-plugin-import] importer.success Importing 59 categories.
[nodebb-plugin-import] importer.success Importing 59/59 categories took: 1.00 seconds
[nodebb-plugin-import] importer.success Importing 4586 topics.
[nodebb-plugin-import] importer.success Importing 4586/4586 topics took: 119.45 seconds
[nodebb-plugin-import] importer.success Importing 58482 posts.
[nodebb-plugin-import] importer.success Importing 58482/58482 posts took: 126.70 seconds

big thanks to @baris for some major optimization.

Oh Also, I added logic to handle mid-import crashes, so you can resume an import now. It is still experimental, but I am targeting the NodeBB 0.5.1 or 0.5.2 release.

[baris]

Compared to 0.5.0, 0.5.1has some serious optimizations everywhere. I think most of the work went into websockets, mongodb, unread and notification system.

[psychobunny]

Damn that's fast. I remember porting a forum with a tenth of those posts and it took like, half an hour or something

[Steve]

@bentael said:

jeezaloo!

dat NodeBB master branch is lightning fast!

@Steve with a combination of [email protected] and [email protected] I was able to import ur DB in less than 5min. Consistently.

[nodebb-plugin-import] importer.success Importing 632 users.
[nodebb-plugin-import] importer.success Importing 632/632 users took: 3.63 seconds
[nodebb-plugin-import] importer.success Importing 59 categories.
[nodebb-plugin-import] importer.success Importing 59/59 categories took: 1.00 seconds
[nodebb-plugin-import] importer.success Importing 4586 topics.
[nodebb-plugin-import] importer.success Importing 4586/4586 topics took: 119.45 seconds
[nodebb-plugin-import] importer.success Importing 58482 posts.
[nodebb-plugin-import] importer.success Importing 58482/58482 posts took: 126.70 seconds

big thanks to @baris for some major optimization.

Oh Also, I added logic to handle mid-import crashes, so you can resume an import now. It is still experimental, but I am targeting the NodeBB 0.5.1 or 0.5.2 release.

looks like i have a late night activity on my hands this evening

amazing job you guys are all doing

[bentael]

I would wait for the release, unless you're just playing with it, I am aware of a couple of bugs

https://github.com/akhoury/nodebb-plugin-import/issues/59
https://github.com/akhoury/nodebb-plugin-import/issues/58

akhoury created this issue in akhoury/nodebb-plugin-import

closed resume is broken #59

akhoury created this issue in akhoury/nodebb-plugin-import

closed get rid of node-gyp dependency #58

[bentael]

Memory issues fixed in 0.2.7. Just imported @Valodim 's SMF large DB against [email protected] release
But had to disable the Markdown plugin during the import only, because of this

Took a good 2 hours on my shitty machine, but that's the price of pagination, at least no server crashing anymore. Start the import and walk away.

Client side page might crash if you enable client side logging and you get a lot of warnings, but dont care, just refresh the page, the process is still running with or without it, but i the future ill add a feature to recover the logs on the page in case that happens.

@Steve start from a fresh [email protected] install, you should be good to go.

[baris]

The two hours might be a lot less if you were on this or later because each new post is parsing the content as markdown which is unnecessary.

[bentael]

true. I just wanted to release the importer against a stable NodeBB release, even though there was some compromise

[Shard]

@bentael Did you get ba chance to beat on that Drupal dump I sent ya?

[bentael]

@Shard not yet man, I've been busy trying to solve memory issues in the importer core. But I will very soon. in the next 2-3 weeks hopefully. so far I got these exporters to implement, Drupal is 2nd on the priority list after Wordpress

https://github.com/akhoury/nodebb-plugin-import/labels/Exporter

[Shard]

Any chance in the mean time?