WordPress to NodeBB

bentael

@a_5mith as far as i know, you can't change the CORE table names, that would break WP. You can only use a custom "table prefix", that's it.

Also. the numbers dont add up, let alone that there are 2 main tables missing, wp_posts and wp_users

@vegetagz6 said:

1600+ posts

wp_9_posts has 105 rows
wp_12_posts has 2 rows

There are no more wp_*posts tables, where are the 1600+ posts?

400 comments

wp_9_comments has 16 rows
wp_12_comment has 1 row
wp_comments has 694 rows <- OK, maybe I should filter

300+ users

There is no wp_users table??

this is a list of all the tables in that schema with their rows count

wp_9_commentmeta: ROWCOUNT=7
wp_9_comments: ROWCOUNT=16
wp_9_layerslider: ROWCOUNT=0
wp_9_links: ROWCOUNT=0
wp_9_options: ROWCOUNT=217
wp_9_postmeta: ROWCOUNT=342
wp_9_posts: ROWCOUNT=105
wp_9_terms: ROWCOUNT=21
wp_9_term_relationships: ROWCOUNT=48
wp_9_term_taxonomy: ROWCOUNT=21
wp_9_wpstickies: ROWCOUNT=0
wp_9_wpstickies_images: ROWCOUNT=0
wp_10_layerslider: ROWCOUNT=0
wp_10_yapbimage: ROWCOUNT=1
wp_11_layerslider: ROWCOUNT=0
wp_11_revslider_settings: ROWCOUNT=0
wp_11_revslider_sliders: ROWCOUNT=0
wp_11_revslider_slides: ROWCOUNT=0
wp_12_commentmeta: ROWCOUNT=0
wp_12_comments: ROWCOUNT=1
wp_12_layerslider: ROWCOUNT=0
wp_12_links: ROWCOUNT=0
wp_12_options: ROWCOUNT=101
wp_12_postmeta: ROWCOUNT=1
wp_12_posts: ROWCOUNT=2
wp_12_terms: ROWCOUNT=1
wp_12_term_relationships: ROWCOUNT=1
wp_12_term_taxonomy: ROWCOUNT=1
wp_aiowps_events: ROWCOUNT=0
wp_aiowps_failed_logins: ROWCOUNT=0
wp_aiowps_global_meta: ROWCOUNT=0
wp_aiowps_login_activity: ROWCOUNT=0
wp_aiowps_login_lockdown: ROWCOUNT=0
wp_author_love: ROWCOUNT=0
wp_bbplike: ROWCOUNT=0
wp_blogs: ROWCOUNT=3
wp_blog_versions: ROWCOUNT=4
wp_bpspro_login_security: ROWCOUNT=0
wp_bpspro_seclog_ignore: ROWCOUNT=0
wp_bp_activity: ROWCOUNT=1,052
wp_bp_activity_meta: ROWCOUNT=582
wp_bp_album: ROWCOUNT=0
wp_bp_album_tags: ROWCOUNT=0
wp_bp_cp_galleries: ROWCOUNT=0
wp_bp_easyalbums_templates: ROWCOUNT=3
wp_bp_friends: ROWCOUNT=112
wp_bp_links: ROWCOUNT=1
wp_bp_links_categories: ROWCOUNT=4
wp_bp_links_linkmeta: ROWCOUNT=1
wp_bp_links_votes: ROWCOUNT=0
wp_bp_messages_messages: ROWCOUNT=3
wp_bp_messages_notices: ROWCOUNT=0
wp_bp_messages_recipients: ROWCOUNT=6
wp_bp_notifications: ROWCOUNT=92
wp_bp_user_blogs: ROWCOUNT=47
wp_bp_user_blogs_blogmeta: ROWCOUNT=3
wp_bp_xprofile_data: ROWCOUNT=4,370
wp_bp_xprofile_fields: ROWCOUNT=5
wp_bp_xprofile_groups: ROWCOUNT=2
wp_bp_xprofile_meta: ROWCOUNT=18
wp_bwwc_btc_addresses: ROWCOUNT=0
wp_cntctfrm_field: ROWCOUNT=13
wp_commentmeta: ROWCOUNT=4,734
wp_comments: ROWCOUNT=694
wp_comments_fbseo: ROWCOUNT=2
wp_cp: ROWCOUNT=30
wp_crumina_slider: ROWCOUNT=0
wp_cr_ips: ROWCOUNT=1
wp_dashboard_chat: ROWCOUNT=9
wp_delucksLoveButton: ROWCOUNT=0
wp_delucksLoveButtonStatistics: ROWCOUNT=0
wp_domain_mapping: ROWCOUNT=0
wp_domain_mapping_logins: ROWCOUNT=0
wp_download_log: ROWCOUNT=507
wp_easy_gallery: ROWCOUNT=0
wp_easy_gallery_images: ROWCOUNT=0
wp_ewwwio_images: ROWCOUNT=34,904
wp_folderfeed_playlist: ROWCOUNT=10
wp_formmaker: ROWCOUNT=2
wp_formmaker_sessions: ROWCOUNT=0
wp_formmaker_submits: ROWCOUNT=0
wp_formmaker_themes: ROWCOUNT=41
wp_formmaker_views: ROWCOUNT=2
wp_frontend_builder_options: ROWCOUNT=124
wp_frontend_builder_pages: ROWCOUNT=0
wp_gallery_albums: ROWCOUNT=0
wp_gallery_pics: ROWCOUNT=0
wp_gallery_settings: ROWCOUNT=1
wp_gdsr_data_article: ROWCOUNT=686
wp_gdsr_data_category: ROWCOUNT=0
wp_gdsr_data_comment: ROWCOUNT=213
wp_gdsr_ips: ROWCOUNT=0
wp_gdsr_moderate: ROWCOUNT=0
wp_gdsr_multis: ROWCOUNT=0
wp_gdsr_multis_data: ROWCOUNT=0
wp_gdsr_multis_trend: ROWCOUNT=0
wp_gdsr_multis_values: ROWCOUNT=0
wp_gdsr_templates: ROWCOUNT=44
wp_gdsr_votes_log: ROWCOUNT=323
wp_gdsr_votes_trend: ROWCOUNT=278
wp_gtabber: ROWCOUNT=10
wp_hmp_playlist: ROWCOUNT=0
wp_hmp_rating: ROWCOUNT=0
wp_html5mp3_items: ROWCOUNT=46
wp_html5mp3_playlist: ROWCOUNT=1
wp_html5mp3_sales: ROWCOUNT=0
wp_ilightbox: ROWCOUNT=11
wp_imsanity: ROWCOUNT=2
wp_itro_plugin_field: ROWCOUNT=1
wp_itro_plugin_option: ROWCOUNT=77
wp_itsec_lockouts: ROWCOUNT=0
wp_itsec_log: ROWCOUNT=0
wp_itsec_temp: ROWCOUNT=0
wp_kklike: ROWCOUNT=0
wp_kklikeuser: ROWCOUNT=0
wp_layerslider: ROWCOUNT=6
wp_like_dislike_counters: ROWCOUNT=1
wp_links: ROWCOUNT=0
wp_lydl_posts: ROWCOUNT=70
wp_lydl_poststimestamp: ROWCOUNT=126
wp_monalisa: ROWCOUNT=17
wp_newsletter: ROWCOUNT=188
wp_newsletter_emails: ROWCOUNT=10
wp_newsletter_stats: ROWCOUNT=156
wp_nggv_settings: ROWCOUNT=15
wp_nggv_votes: ROWCOUNT=0
wp_ngg_album: ROWCOUNT=1
wp_ngg_gallery: ROWCOUNT=21
wp_ngg_pictures: ROWCOUNT=677
wp_nh_locations: ROWCOUNT=0
wp_nxs_log: ROWCOUNT=0
wp_oauth2_access_tokens: ROWCOUNT=0
wp_oauth2_auth_codes: ROWCOUNT=0
wp_oauth2_clients: ROWCOUNT=1
wp_oauth2_options: ROWCOUNT=0
wp_oauth2_refresh_tokens: ROWCOUNT=0
wp_om_hits_log: ROWCOUNT=1,207
wp_options: ROWCOUNT=5,521
wp_option_tree: ROWCOUNT=38

bentael

I installed a fresh WP 4 install, and here's how a fresh DB looks like - i can write an exporter for this, anything non-standard needs its own exporter, unless there is some logical code path we can take for each case, then yea

Screen Shot 2014-10-27 at 11.47.57 PM.png

a_5mith

@bentael said:

I installed a fresh WP 4 install, and here's how a fresh DB looks like - i can write an exporter for this, anything non-standard needs its own exporter, unless there is some logical code path we can take for each case, then yea

I'd say that would be the most logical approach. I'm pretty sure the relevant pieces could be extracted from a custom Wordpress install, and placed into a clean one.

Meliodas

Hmm, ya that database was part of a multi site install and there are have been used some plugins which added lot more tables. But like a_5mith mentioned, in case of old sites likes these which have modified databases, is not it possible to extract the required ones alone?

Another idea which struck me was, if such databases will need their own exporter, why not, in my case, I transfer the required wordpress database of this site to another simple new install of non multi site wordpress, and then export it to nodebb. Because wordpress to wordpress exporters usually figure all out all type of complicated databases. So do you think such two stage exports will do the trick for multi site or complicated wordpress databases?

Am asking this because, in our case, as you saw, the database has been modified a lot, because we used many plugins, some which I made and some from codecanyon, which introduced some really necessary functions for us.
@bentael

a_5mith

@vegetagz6 The exporter would just look for the required data, so users, comments, posts, categories and maybe the tags if it's not too difficult. But your database doesn't include the users, so it won't work as is. You will need to convert your multi site db, back to a clean DB. Other tables are ignored, the import plugin only looks for specific table names in the first place. The plugin would work fine for users with a largely modified forum. But it seems that a multi site import is another step entirely.

If I'm not mistaken, I believe wp_users would be on the "master" install of Wordpress, then each subsidiary Wordpress site (wp_1_ , wp_2_ , etc) just contains the information for that wordpress. The user accounts would be verified against the "master database", which leads to an issue when you want to take one of those multi site installs, and import it into NodeBB. You would need to merge the master DB with wp_9_, in a way that didn't cause the loss of any posts, then use the importer.

Meliodas

Ok converting multi site back to normal wordpress is easy, that can be done.

But as an ALTERNATE option, won't this plugin work if I exported the required site from a multi site to a normal wordpress installation using wordpress exporter and then go for nodebb export? I think it should work, because a fresh WordPress installation only contains the core tables. So once the users, posts and comments are transferred to a fresh wordpress installation, we can use the nodebb plugin for the final export.

a_5mith

@vegetagz6 said:

Ok converting multi site back to normal wordpress is easy, that can be done.

But as an ALTERNATE option, won't this plugin work if I exported the required site from a multi site to a normal wordpress installation using wordpress exporter and then go for nodebb export? I think it should work, because a fresh WordPress installation only contains the core tables. So once the users, posts and comments are transferred to a fresh wordpress installation, we can use the nodebb plugin for the final export.

What I said and what you said were the same thing, so yes. If there's a tool that does it for you, then that would be fine. As long as the final database has all of the tables in this image and they're populated with the right number of entries:

this one

bentael

I don't even mind handling a "multi-site" WP DB, in one exporter, as long as everything is in a single Database, not multi (that would make it really hairy)

I can add an advanced-custom-config that accepts a extra custom JSON for the exporter in the UI, then plumbs it through to the exporter module, so something like

exporterConfig = {
  dbhost: 'localhost',
  dbname: 'wp'
  dbuser: 'user',
  dbpassword: 'password',
  tablePrefix: 'wp_',
  custom: {
    // anything you want here, 
    // that would be a result of a JSON.parse(/* what's on an advanced UI field */ )
  
    // in the case of the wp exporter,  a new custom option is introduced to specify which WP site you want to target
    // the UI value would be string '{ "wpsite": "1" }'
    wpsite: "1,2" // that means, site 1 and site 2,   
    // For each Site, the exporter would create a new NodeBB Parent-Category, that's doable too
    // wpsite: "all", // all sites, you guessed it
  }
}

however, my problem is still the same @vegetagz6,

where are the users?

where are the posts?

is it possible that you might have another database?

a_5mith

@bentael I would try and find someone else with a standard WP site, as it's looking like there's at least 12 databases here, if the schema of database numbering is anything to go by.

bentael

okay.png,

@vegetagz6 unless you can figure out the answers to my questions, I don't think i can help you much, sorry

anyone else has a standard WP DB with some data?

a_5mith

Mentioning previous people that wanted this plugin

@Flexi-Hackify , @Tan-Nguyen , @John-Smith , @Hawon-Nguyen

Meliodas

Ok am sending u a standard database, which has all the standard tables like wp_comments, wp_users etc. Check your pm and lemme know

Ya the previous database was heavily modified, in the sense that it was part of a multisite installation, and belonged to one of the most active site I have.

Meliodas

@a_5mith any updates from Bentael on this plugin, yet?

bentael

@vegetagz6 no updates yet - maybe next week, sorry ive been very busy combined with very sick for the past 2 weeks.

bentael

wordpress exporter released,
https://www.npmjs.org/package/nodebb-plugin-import-wordpress

@vegetagz6
I only tested against your "dota" db, looks good so far except that you only use 1 out of 2 categories. Your "droid" db had only 2 users..

Needs more testing of course,

but can't do much without many DB Dumps,

I also need to export tags for topics but I need more dumps to test with.

bentael

@John-Smith
@Hawon-Nguyen
@Flexi-Hackify
@Tan-Nguyen

read 1 post up
To use it, you just need to install https://github.com/akhoury/nodebb-plugin-import and follow instructions in the Admin plugin page

Tan Nguyen

@bentael Thank you very much for all of your hard works. What happen to the old post link. I mean will it automatically redirect to the new link (nodebb)?

bentael

@Tan-Nguyen nothing redirects automatically, you have to set something in the middle to redirect. The import plugin will provide a map from old to new.

Read this note: https://github.com/akhoury/nodebb-plugin-import#imported-now-what focus on the redirect section

bentael

[email protected] now supports importing topic tags - to use it, just use the latest nodebb-plugin-import, and select the nodebb-plugin-import-wordpress from the exporters list (or paste it in the input field) and it will install the latest. As a precaution, uninstall the previous version first
npm uninstall nodebb-plugin-import-wordpress

Right now I query all tags and create a map using the topic ids, I know you don't care, but the impact of this is that if you have millions or maybe hundreds of thousands of tags, depending on your machine, you might run into memory issues

are u a sql guru?

If you are a MySQL guru and can help optimizing the query, try to answer this question.
http://wordpress.stackexchange.com/questions/169863/wordpress-custom-sql-query-get-all-posts-with-category-id-and-a-concated-list

if you need to look at the exporter code, it has more fields to select here than the stackexchange question because I wanted to simplify the question as much as possible.
https://github.com/a5mith/nodebb-plugin-import-wordpress/blob/master/index.js#L127-L233

bentael

even more, now the wordpress gallery shortcodes are supported, with the old post ids, i.e. [gallery ids="123,456,789"] shortcodes can be converted to, either, <img> tags, if you want a gallery plugin, you can then use @psychobunny's nodebb-plugin-gallery, or you can just convert the ids to full URLs which then you can use within your own plugin.

read this section for more details
https://github.com/a5mith/nodebb-plugin-import-wordpress#gallery-shortcode-note