SMF to NodeBB

psychobunny

I hear the SMF BBCode is a total pain in the ass. @bentael I don't mind helping out with the regex on this if that means you can focus on the other importers on your spare time instead

For your forum I think you did a lot of pre-formatting in SQL itself right @a_5mith ?

a_5mith

@psychobunny said:

For your forum I think you did a lot of pre-formatting in SQL itself right @a_5mith ?

It's the quotes that are awful, the rest of the BBcode is okay. I think Bentael sorted the bbcode for quotes in the older version.

UPDATE smf_messages SET body = REPLACE(body, '[img]', '');
UPDATE smf_messages SET body = REPLACE(body, '[/img]', '');
UPDATE smf_messages SET body = REPLACE(body, '[IMG]', '');
UPDATE smf_messages SET body = REPLACE(body, '[/IMG]', '');
UPDATE smf_messages SET body = REPLACE(body, '[sup]', '');
UPDATE smf_messages SET body = REPLACE(body, '[/sup]', '');
UPDATE smf_messages SET body = REPLACE(body, '[move]', '');
UPDATE smf_messages SET body = REPLACE(body, '[/move]', '');
UPDATE smf_messages SET body = REPLACE(body, '[spoiler', '>! ');
UPDATE smf_messages SET body = REPLACE(body, '[/spoiler]', '');
UPDATE smf_messages SET body = REPLACE(body, '[youtube]http://m.youtube.com/watch?v=', 'https://www.youtube.com/watch?v=');
UPDATE smf_messages SET body = REPLACE(body, '[youtube]', 'https://www.youtube.com/watch?v=');
UPDATE smf_messages SET body = REPLACE(body, '[/youtube]', '');
UPDATE smf_messages SET body = REPLACE(body, '[b]', '**');
UPDATE smf_messages SET body = REPLACE(body, '[/b]', '**');
UPDATE smf_messages SET body = REPLACE(body, '[i]', '*');
UPDATE smf_messages SET body = REPLACE(body, '[/i]', '*');
UPDATE smf_members SET signature = REPLACE(signature, '[move]', '');
UPDATE smf_members SET signature = REPLACE(signature, '[/move]', '');
UPDATE smf_members SET signature = REPLACE(signature, '[b]', '**');
UPDATE smf_members SET signature = REPLACE(signature, '[/b]', '**');
UPDATE smf_members SET signature = REPLACE(signature, '[img]', '');
UPDATE smf_members SET signature = REPLACE(signature, '[/img]', '');
UPDATE smf_members SET signature = REPLACE(signature, '[IMG]', '');
UPDATE smf_members SET signature = REPLACE(signature, '[/IMG]', '');

Then did another one after to fix the youtube embeds that duplicated.

 UPDATE smf_messages SET body = REPLACE(body, 'https://www.youtube.com/watch?v=https://www.youtube.com/watch?v=', 'https://www.youtube.com/watch?v=');
UPDATE smf_messages SET body = REPLACE(body, 'https://www.youtube.com/watch?v=http://www.youtube.com/watch?v=', 'https://www.youtube.com/watch?v=');
UPDATE smf_messages SET body = REPLACE(body, 'https://www.youtube.com/watch?v=http://www.youtube.com/watch?v=', 'https://www.youtube.com/watch?v=');
UPDATE smf_messages SET body = REPLACE(body, 'https://www.youtube.com/watch?v=https://www.youtube.com/watch?v=', 'http://www.youtube.com/watch?v=');

psychobunny

Oh okay this doesn't look too bad. Maybe I was thinking of another forum when I said the BBCode is a pain. In that case

also, did you use the "BBCode to Markdown" option before importing?

I would imagine this would take care of that

a_5mith

@psychobunny said:

Oh okay this doesn't look too bad. Maybe I was thinking of another forum when I said the BBCode is a pain. In that case

Well the BBcode for quotes is

[quote author=author link=topic=55.msg398#msg398 date=1384327984]<br />POST CONTENT[/quote]

I think it was phpbb that was a pain, as it appends all BBcode with an ID. so [b]text[/b] would be [b id=randonstring]text[/b id=samerandomstring]

psychobunny

Ah yes. And I've already written the plugin for that so clearly I have the memory of a goldfish

bentael

@a_5mith @psychobunny
what do you guys say we start putting together a list of all the bbcode tags?

the module that bbcode-to-markdown is using, bbcodejs (which i forked to support nodejs) already supports plenty of tags
https://github.com/akhoury/bbcodejs/blob/master/src/js/tags.js#L416-L464

I've added many that it was missing (including quotes and few fancy ones)
https://github.com/akhoury/bbcode-to-markdown/blob/master/newTags.js#L4-L53

but really, I add them as I find them, I search all the internets, there is no definitve list of all bbcode tags or a syntax standards anywhere

a_5mith

@bentael I can strip the full list out of SMF's core files if you want? There's about 40 in there. I only posted the above ones as they were what were used most. That should give you the majority of BBcode.

bentael

k awesome.

a_5mith

@bentael said:

k awesome.

array( 'tag' => 'abbr', 'type' => 'unparsed_equals', 'before' => - Pastebin.com

Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.

Pastebin (pastebin.com)

The 'tag' is what's wrapped in the square brackets. A lot of these are probably never going to be needed. I was going to remove the code portion, but noticed that they're still actively supporting IE4...

Silvina

@bentael @psychobunny @a_5mith

Hi All,

Did an import of my forum with the following settings:

No other plugins enabled.

The result: http://nodebb.krng.me/

From the category previews, you will be able to see some of the posts with BBCODE not processed.

a_5mith

Hmm, I've spotted an issue slightly worse than the BBcode, but I'll send it to akhoury privately just to make sure.

The conversion should be bbcode-to-md.

bentael

@Silvina looks like you're using nodebb-plugin-import#master, please use the published 0.2.1 version, I am in process of refactoring many parts and the master version is not stable.

Silvina

@bentael

Hi do you mean this (https://github.com/akhoury/nodebb-plugin-import/tree/0.2.0)? I can't seem to find the 0.2.1 version anywhere on github

bentael

it doesn't need to be github, just use npm:

# change directory to the NodeBB root directory
cd /path/to/NodeBB

# remove the one you already have
rm -rf node_modules/nodebb-plugin-import

# install from npm
npm install nodebb-plugin-import

this will install the latest published one, 0.2.1

Note: I am publishing a slightly better version late tonight, if you want to wait, if not, just install 0.2.1 and play with it, I am aware of few bugs that I will be fixing in tonight's release

Silvina

@bentael Thanks.
Just did an import using the latest version (0.2.1)

Much better on the BBCode to Markdown conversion, though still have some issues.

eg:
http://nodebb.krng.me/topic/165/star-citizen-squadron-42-edge-foundation-asia-fleet
http://nodebb.krng.me/topic/339/free-games-put-and-get-your-free-games-here/2
http://nodebb.krng.me/topic/216/elite-dangerous/2

Will wait for your next release Really appreciate you efforts in this! Thanks in advance

bentael

do you have the markdown plugin enabled?

Silvina

Nope, let me enable it and try again.

Enabled: Working great now.

There are some posts which i believe i will need to enable some other plugins.
http://nodebb.krng.me/topic/255/item-level-90-gear-comparison
Will test further Thanks a lot man!

Silvina

@bentael @a_5mith
Noticed a bug, nothing to do with BBCode conversion.

I have a few threads that have more than 50+ posts, but all the imported threads only have 2 posts max.
eg
http://nodebb.krng.me/topic/165/star-citizen-squadron-42-edge-foundation-asia-fleet
http://nodebb.krng.me/topic/339/free-games-put-and-get-your-free-games-here

bentael

new version delayed 1 or 2 days, I keep finding more work to get done.

Silvina

no worries on that.