SMF to NodeBB

Saleem

Is there an SMF to NodeBB migration tool? I saw a 10-month old thread on some progress, but it's unclear to me. Thanks!

My forum is 250k posts

a_5mith

@Saleem SMF plugin does indeed work.

Use this: https://github.com/a5mith/nodebb-plugin-import-smf

Follow the instructions in this ReadMe: https://github.com/akhoury/nodebb-plugin-import-ubb

Then use this to get your posts into NodeBB: https://github.com/akhoury/nodebb-plugin-import

Might take a while. I've transferred 28,000 posts from SMF without issue.

Silvina

I agree with the OP. The import is working, however, as passwords are not imported, every user had to reset their passwords.

a_5mith

A solution to porting passwords over is possibly in the works.

IP.Board Migration Tool · Issue #2008 · NodeBB/NodeBB

Re: https://community.nodebb.org/topic/2273/still-not-got-ipb-nodebb-plugin-x Going to see if I can put this up on BountySource...

GitHub (github.com)

Silvina

Thats a very good news!

Where can i monitor the progress of this ?

Silvina

Is the importer working for "nodebb v0.5.0-4" ?

My nodebb instance could not start the moment i added the import plugins.

a_5mith

@Silvina Right, which plugin are you trying to run? nodebb-plugin-import? If you run ./nodebb dev what's the error?

Silvina

@a_5mith thanks for the quick reply. let me try again with a clean install.

With regards to the nodebb-plugin-import, how do i ensure i am using the 0.20 branch? or i can continue to use what is in the default install of nodebb?

a_5mith

@Silvina said:

With regards to the nodebb-plugin-import, how do i ensure i am using the 0.20 branch? or i can continue to use what is in the default install of nodebb?

It would appear that @bentael needs to push the latest version to npm, you can install 0.2.0 of nodebb-plugin-import-smf by running git clone https://github.com/a5mith/nodebb-plugin-import-smf.git inside your node_modules folder, then run cd nodebb-plugin-import-smf then npm install.

I think bentael is about, so should respond with definite instructions soon.

Silvina

Managed to did an import!

However, i realised none of the BBCODE conversion is working, either via the import scripts or post import plugins

bentael

@Silvina BBCode has been a nightmare to me, there is no real standards, even if there is, every forum software modify and use it differently.

I am trying to combine all the converters and gather all the tags together in this module https://github.com/akhoury/bbcode-to-markdown which is what the -import plugin is using.

If you can share with me samples of what did not convert, I can try to add the missing tags to the module.

bentael

also, did you use the "BBCode to Markdown" option before importing?

psychobunny

I hear the SMF BBCode is a total pain in the ass. @bentael I don't mind helping out with the regex on this if that means you can focus on the other importers on your spare time instead

For your forum I think you did a lot of pre-formatting in SQL itself right @a_5mith ?

a_5mith

@psychobunny said:

For your forum I think you did a lot of pre-formatting in SQL itself right @a_5mith ?

It's the quotes that are awful, the rest of the BBcode is okay. I think Bentael sorted the bbcode for quotes in the older version.

UPDATE smf_messages SET body = REPLACE(body, '[img]', '');
UPDATE smf_messages SET body = REPLACE(body, '[/img]', '');
UPDATE smf_messages SET body = REPLACE(body, '[IMG]', '');
UPDATE smf_messages SET body = REPLACE(body, '[/IMG]', '');
UPDATE smf_messages SET body = REPLACE(body, '[sup]', '');
UPDATE smf_messages SET body = REPLACE(body, '[/sup]', '');
UPDATE smf_messages SET body = REPLACE(body, '[move]', '');
UPDATE smf_messages SET body = REPLACE(body, '[/move]', '');
UPDATE smf_messages SET body = REPLACE(body, '[spoiler', '>! ');
UPDATE smf_messages SET body = REPLACE(body, '[/spoiler]', '');
UPDATE smf_messages SET body = REPLACE(body, '[youtube]http://m.youtube.com/watch?v=', 'https://www.youtube.com/watch?v=');
UPDATE smf_messages SET body = REPLACE(body, '[youtube]', 'https://www.youtube.com/watch?v=');
UPDATE smf_messages SET body = REPLACE(body, '[/youtube]', '');
UPDATE smf_messages SET body = REPLACE(body, '[b]', '**');
UPDATE smf_messages SET body = REPLACE(body, '[/b]', '**');
UPDATE smf_messages SET body = REPLACE(body, '[i]', '*');
UPDATE smf_messages SET body = REPLACE(body, '[/i]', '*');
UPDATE smf_members SET signature = REPLACE(signature, '[move]', '');
UPDATE smf_members SET signature = REPLACE(signature, '[/move]', '');
UPDATE smf_members SET signature = REPLACE(signature, '[b]', '**');
UPDATE smf_members SET signature = REPLACE(signature, '[/b]', '**');
UPDATE smf_members SET signature = REPLACE(signature, '[img]', '');
UPDATE smf_members SET signature = REPLACE(signature, '[/img]', '');
UPDATE smf_members SET signature = REPLACE(signature, '[IMG]', '');
UPDATE smf_members SET signature = REPLACE(signature, '[/IMG]', '');

Then did another one after to fix the youtube embeds that duplicated.

 UPDATE smf_messages SET body = REPLACE(body, 'https://www.youtube.com/watch?v=https://www.youtube.com/watch?v=', 'https://www.youtube.com/watch?v=');
UPDATE smf_messages SET body = REPLACE(body, 'https://www.youtube.com/watch?v=http://www.youtube.com/watch?v=', 'https://www.youtube.com/watch?v=');
UPDATE smf_messages SET body = REPLACE(body, 'https://www.youtube.com/watch?v=http://www.youtube.com/watch?v=', 'https://www.youtube.com/watch?v=');
UPDATE smf_messages SET body = REPLACE(body, 'https://www.youtube.com/watch?v=https://www.youtube.com/watch?v=', 'http://www.youtube.com/watch?v=');

psychobunny

Oh okay this doesn't look too bad. Maybe I was thinking of another forum when I said the BBCode is a pain. In that case

also, did you use the "BBCode to Markdown" option before importing?

I would imagine this would take care of that

a_5mith

@psychobunny said:

Oh okay this doesn't look too bad. Maybe I was thinking of another forum when I said the BBCode is a pain. In that case

Well the BBcode for quotes is

[quote author=author link=topic=55.msg398#msg398 date=1384327984]<br />POST CONTENT[/quote]

I think it was phpbb that was a pain, as it appends all BBcode with an ID. so [b]text[/b] would be [b id=randonstring]text[/b id=samerandomstring]

psychobunny

Ah yes. And I've already written the plugin for that so clearly I have the memory of a goldfish

bentael

@a_5mith @psychobunny
what do you guys say we start putting together a list of all the bbcode tags?

the module that bbcode-to-markdown is using, bbcodejs (which i forked to support nodejs) already supports plenty of tags
https://github.com/akhoury/bbcodejs/blob/master/src/js/tags.js#L416-L464

I've added many that it was missing (including quotes and few fancy ones)
https://github.com/akhoury/bbcode-to-markdown/blob/master/newTags.js#L4-L53

but really, I add them as I find them, I search all the internets, there is no definitve list of all bbcode tags or a syntax standards anywhere

a_5mith

@bentael I can strip the full list out of SMF's core files if you want? There's about 40 in there. I only posted the above ones as they were what were used most. That should give you the majority of BBcode.

bentael

k awesome.