• Plugin & Theme Dev

    @vegetagz6 holy crap, that WP DB is too.. weird... well, I don't understand few things:

    According to this official WP guide, http://codex.wordpress.org/Database_Description (which only goes up to 3.8, but heck, close enough) the WP Database schema should be, fairly, simple.

    Looking at the guide, I can definitely write an exporter for it in few hours, but that example you sent me threw me off.

    For example, based on the schema below

    • What the hell are the wp_{NUMBER}_{TABLENAME} such as wp_9_posts/wp_9_comments and wp_12_posts/wp_12_comments
    • Are these some attempt of pagination? migration results? that schema doesn't even have a regular wp_posts table?
    • Where are the "other" numbers, such as 1, 2, 3, .... 8?
    • am I suppose to scan the schema for all of these relevant tables and join them?

    I understand you might have some plugins that might edit the schema, but this is too non-standard, I don't think i can write a general purpose WP exporter based on this schema.

    Even reading the WP DB handler source seems pretty straight forward. https://github.com/WordPress/WordPress/blob/master/wp-includes/wp-db.php

    I am definitely missing something and I need some guidance from a WP guru.

    wp_9_commentmeta
    wp_9_comments
    wp_9_layerslider
    wp_9_links
    wp_9_options
    wp_9_postmeta
    wp_9_posts
    wp_9_terms
    wp_9_term_relationships
    wp_9_term_taxonomy
    wp_9_wpstickies
    wp_9_wpstickies_images
    wp_10_layerslider
    wp_10_yapbimage
    wp_11_layerslider
    wp_11_revslider_settings
    wp_11_revslider_sliders
    wp_11_revslider_slides
    wp_12_commentmeta
    wp_12_comments
    wp_12_layerslider
    wp_12_links
    wp_12_options
    wp_12_postmeta
    wp_12_posts
    wp_12_terms
    wp_12_term_relationships
    wp_12_term_taxonomy
    wp_aiowps_events
    wp_aiowps_failed_logins
    wp_aiowps_global_meta
    wp_aiowps_login_activity
    wp_aiowps_login_lockdown
    wp_author_love
    wp_bbplike
    wp_blogs
    wp_blog_versions
    wp_bpspro_login_security
    wp_bpspro_seclog_ignore
    wp_bp_activity
    wp_bp_activity_meta
    wp_bp_album
    wp_bp_album_tags
    wp_bp_cp_galleries
    wp_bp_easyalbums_templates
    wp_bp_friends
    wp_bp_links
    wp_bp_links_categories
    wp_bp_links_linkmeta
    wp_bp_links_votes
    wp_bp_messages_messages
    wp_bp_messages_notices
    wp_bp_messages_recipients
    wp_bp_notifications
    wp_bp_user_blogs
    wp_bp_user_blogs_blogmeta
    wp_bp_xprofile_data
    wp_bp_xprofile_fields
    wp_bp_xprofile_groups
    wp_bp_xprofile_meta
    wp_bwwc_btc_addresses
    wp_cntctfrm_field
    wp_commentmeta
    wp_comments
    wp_comments_fbseo
    wp_cp
    wp_crumina_slider
    wp_cr_ips
    wp_dashboard_chat
    wp_delucksLoveButton
    wp_delucksLoveButtonStatistics
    wp_domain_mapping
    wp_domain_mapping_logins
    wp_download_log
    wp_easy_gallery
    wp_easy_gallery_images
    wp_ewwwio_images
    wp_folderfeed_playlist
    wp_formmaker
    wp_formmaker_sessions
    wp_formmaker_submits
    wp_formmaker_themes
    wp_formmaker_views
    wp_frontend_builder_options
    wp_frontend_builder_pages
    wp_gallery_albums
    wp_gallery_pics
    wp_gallery_settings
    wp_gdsr_data_article
    wp_gdsr_data_category
    wp_gdsr_data_comment
    wp_gdsr_ips
    wp_gdsr_moderate
    wp_gdsr_multis
    wp_gdsr_multis_data
    wp_gdsr_multis_trend
    wp_gdsr_multis_values
    wp_gdsr_templates
    wp_gdsr_votes_log
    wp_gdsr_votes_trend
    wp_gtabber
    wp_hmp_playlist
    wp_hmp_rating
    wp_html5mp3_items
    wp_html5mp3_playlist
    wp_html5mp3_sales
    wp_ilightbox
    wp_imsanity
    wp_itro_plugin_field
    wp_itro_plugin_option
    wp_itsec_lockouts
    wp_itsec_log
    wp_itsec_temp
    wp_kklike
    wp_kklikeuser
    wp_layerslider
    wp_like_dislike_counters
    wp_links
    wp_lydl_posts
    wp_lydl_poststimestamp
    wp_monalisa
    wp_newsletter
    wp_newsletter_emails
    wp_newsletter_stats
    wp_nggv_settings
    wp_nggv_votes
    wp_ngg_album
    wp_ngg_gallery
    wp_ngg_pictures
    wp_nh_locations
    wp_nxs_log
    wp_oauth2_access_tokens
    wp_oauth2_auth_codes
    wp_oauth2_clients
    wp_oauth2_options
    wp_oauth2_refresh_tokens
    wp_om_hits_log
    wp_options
    wp_option_tree
    

  • @bentael Could it not just be the table names? Rather than just wp_comments etc, owner of that database may have used wp_9_comments. Alternatively, it could be part of a multi site install. There's a data plugin for Wordpress that will create X number of whatever you need, comments, posts, etc. You could probably compare them, see how they differ.

  • Plugin & Theme Dev

    @a_5mith as far as i know, you can't change the CORE table names, that would break WP. You can only use a custom "table prefix", that's it.

    Also. the numbers dont add up, let alone that there are 2 main tables missing, wp_posts and wp_users

    @vegetagz6 said:

    1600+ posts

    wp_9_posts has 105 rows
    wp_12_posts has 2 rows

    There are no more wp_*posts tables, where are the 1600+ posts?

    400 comments

    wp_9_comments has 16 rows
    wp_12_comment has 1 row
    wp_comments has 694 rows <- OK, maybe I should filter

    300+ users

    There is no wp_users table??

    this is a list of all the tables in that schema with their rows count

    wp_9_commentmeta: ROWCOUNT=7
    wp_9_comments: ROWCOUNT=16
    wp_9_layerslider: ROWCOUNT=0
    wp_9_links: ROWCOUNT=0
    wp_9_options: ROWCOUNT=217
    wp_9_postmeta: ROWCOUNT=342
    wp_9_posts: ROWCOUNT=105
    wp_9_terms: ROWCOUNT=21
    wp_9_term_relationships: ROWCOUNT=48
    wp_9_term_taxonomy: ROWCOUNT=21
    wp_9_wpstickies: ROWCOUNT=0
    wp_9_wpstickies_images: ROWCOUNT=0
    wp_10_layerslider: ROWCOUNT=0
    wp_10_yapbimage: ROWCOUNT=1
    wp_11_layerslider: ROWCOUNT=0
    wp_11_revslider_settings: ROWCOUNT=0
    wp_11_revslider_sliders: ROWCOUNT=0
    wp_11_revslider_slides: ROWCOUNT=0
    wp_12_commentmeta: ROWCOUNT=0
    wp_12_comments: ROWCOUNT=1
    wp_12_layerslider: ROWCOUNT=0
    wp_12_links: ROWCOUNT=0
    wp_12_options: ROWCOUNT=101
    wp_12_postmeta: ROWCOUNT=1
    wp_12_posts: ROWCOUNT=2
    wp_12_terms: ROWCOUNT=1
    wp_12_term_relationships: ROWCOUNT=1
    wp_12_term_taxonomy: ROWCOUNT=1
    wp_aiowps_events: ROWCOUNT=0
    wp_aiowps_failed_logins: ROWCOUNT=0
    wp_aiowps_global_meta: ROWCOUNT=0
    wp_aiowps_login_activity: ROWCOUNT=0
    wp_aiowps_login_lockdown: ROWCOUNT=0
    wp_author_love: ROWCOUNT=0
    wp_bbplike: ROWCOUNT=0
    wp_blogs: ROWCOUNT=3
    wp_blog_versions: ROWCOUNT=4
    wp_bpspro_login_security: ROWCOUNT=0
    wp_bpspro_seclog_ignore: ROWCOUNT=0
    wp_bp_activity: ROWCOUNT=1,052
    wp_bp_activity_meta: ROWCOUNT=582
    wp_bp_album: ROWCOUNT=0
    wp_bp_album_tags: ROWCOUNT=0
    wp_bp_cp_galleries: ROWCOUNT=0
    wp_bp_easyalbums_templates: ROWCOUNT=3
    wp_bp_friends: ROWCOUNT=112
    wp_bp_links: ROWCOUNT=1
    wp_bp_links_categories: ROWCOUNT=4
    wp_bp_links_linkmeta: ROWCOUNT=1
    wp_bp_links_votes: ROWCOUNT=0
    wp_bp_messages_messages: ROWCOUNT=3
    wp_bp_messages_notices: ROWCOUNT=0
    wp_bp_messages_recipients: ROWCOUNT=6
    wp_bp_notifications: ROWCOUNT=92
    wp_bp_user_blogs: ROWCOUNT=47
    wp_bp_user_blogs_blogmeta: ROWCOUNT=3
    wp_bp_xprofile_data: ROWCOUNT=4,370
    wp_bp_xprofile_fields: ROWCOUNT=5
    wp_bp_xprofile_groups: ROWCOUNT=2
    wp_bp_xprofile_meta: ROWCOUNT=18
    wp_bwwc_btc_addresses: ROWCOUNT=0
    wp_cntctfrm_field: ROWCOUNT=13
    wp_commentmeta: ROWCOUNT=4,734
    wp_comments: ROWCOUNT=694
    wp_comments_fbseo: ROWCOUNT=2
    wp_cp: ROWCOUNT=30
    wp_crumina_slider: ROWCOUNT=0
    wp_cr_ips: ROWCOUNT=1
    wp_dashboard_chat: ROWCOUNT=9
    wp_delucksLoveButton: ROWCOUNT=0
    wp_delucksLoveButtonStatistics: ROWCOUNT=0
    wp_domain_mapping: ROWCOUNT=0
    wp_domain_mapping_logins: ROWCOUNT=0
    wp_download_log: ROWCOUNT=507
    wp_easy_gallery: ROWCOUNT=0
    wp_easy_gallery_images: ROWCOUNT=0
    wp_ewwwio_images: ROWCOUNT=34,904
    wp_folderfeed_playlist: ROWCOUNT=10
    wp_formmaker: ROWCOUNT=2
    wp_formmaker_sessions: ROWCOUNT=0
    wp_formmaker_submits: ROWCOUNT=0
    wp_formmaker_themes: ROWCOUNT=41
    wp_formmaker_views: ROWCOUNT=2
    wp_frontend_builder_options: ROWCOUNT=124
    wp_frontend_builder_pages: ROWCOUNT=0
    wp_gallery_albums: ROWCOUNT=0
    wp_gallery_pics: ROWCOUNT=0
    wp_gallery_settings: ROWCOUNT=1
    wp_gdsr_data_article: ROWCOUNT=686
    wp_gdsr_data_category: ROWCOUNT=0
    wp_gdsr_data_comment: ROWCOUNT=213
    wp_gdsr_ips: ROWCOUNT=0
    wp_gdsr_moderate: ROWCOUNT=0
    wp_gdsr_multis: ROWCOUNT=0
    wp_gdsr_multis_data: ROWCOUNT=0
    wp_gdsr_multis_trend: ROWCOUNT=0
    wp_gdsr_multis_values: ROWCOUNT=0
    wp_gdsr_templates: ROWCOUNT=44
    wp_gdsr_votes_log: ROWCOUNT=323
    wp_gdsr_votes_trend: ROWCOUNT=278
    wp_gtabber: ROWCOUNT=10
    wp_hmp_playlist: ROWCOUNT=0
    wp_hmp_rating: ROWCOUNT=0
    wp_html5mp3_items: ROWCOUNT=46
    wp_html5mp3_playlist: ROWCOUNT=1
    wp_html5mp3_sales: ROWCOUNT=0
    wp_ilightbox: ROWCOUNT=11
    wp_imsanity: ROWCOUNT=2
    wp_itro_plugin_field: ROWCOUNT=1
    wp_itro_plugin_option: ROWCOUNT=77
    wp_itsec_lockouts: ROWCOUNT=0
    wp_itsec_log: ROWCOUNT=0
    wp_itsec_temp: ROWCOUNT=0
    wp_kklike: ROWCOUNT=0
    wp_kklikeuser: ROWCOUNT=0
    wp_layerslider: ROWCOUNT=6
    wp_like_dislike_counters: ROWCOUNT=1
    wp_links: ROWCOUNT=0
    wp_lydl_posts: ROWCOUNT=70
    wp_lydl_poststimestamp: ROWCOUNT=126
    wp_monalisa: ROWCOUNT=17
    wp_newsletter: ROWCOUNT=188
    wp_newsletter_emails: ROWCOUNT=10
    wp_newsletter_stats: ROWCOUNT=156
    wp_nggv_settings: ROWCOUNT=15
    wp_nggv_votes: ROWCOUNT=0
    wp_ngg_album: ROWCOUNT=1
    wp_ngg_gallery: ROWCOUNT=21
    wp_ngg_pictures: ROWCOUNT=677
    wp_nh_locations: ROWCOUNT=0
    wp_nxs_log: ROWCOUNT=0
    wp_oauth2_access_tokens: ROWCOUNT=0
    wp_oauth2_auth_codes: ROWCOUNT=0
    wp_oauth2_clients: ROWCOUNT=1
    wp_oauth2_options: ROWCOUNT=0
    wp_oauth2_refresh_tokens: ROWCOUNT=0
    wp_om_hits_log: ROWCOUNT=1,207
    wp_options: ROWCOUNT=5,521
    wp_option_tree: ROWCOUNT=38
    
  • Plugin & Theme Dev

    I installed a fresh WP 4 install, and here's how a fresh DB looks like - i can write an exporter for this, anything non-standard needs its own exporter, unless there is some logical code path we can take for each case, then yea

    Screen Shot 2014-10-27 at 11.47.57 PM.png


  • @bentael said:

    I installed a fresh WP 4 install, and here's how a fresh DB looks like - i can write an exporter for this, anything non-standard needs its own exporter, unless there is some logical code path we can take for each case, then yea

    Screen Shot 2014-10-27 at 11.47.57 PM.png

    I'd say that would be the most logical approach. I'm pretty sure the relevant pieces could be extracted from a custom Wordpress install, and placed into a clean one.


  • Hmm, ya that database was part of a multi site install and there are have been used some plugins which added lot more tables. But like a_5mith mentioned, in case of old sites likes these which have modified databases, is not it possible to extract the required ones alone?

    Another idea which struck me was, if such databases will need their own exporter, why not, in my case, I transfer the required wordpress database of this site to another simple new install of non multi site wordpress, and then export it to nodebb. Because wordpress to wordpress exporters usually figure all out all type of complicated databases. So do you think such two stage exports will do the trick for multi site or complicated wordpress databases?

    Am asking this because, in our case, as you saw, the database has been modified a lot, because we used many plugins, some which I made and some from codecanyon, which introduced some really necessary functions for us.
    @bentael


  • @vegetagz6 The exporter would just look for the required data, so users, comments, posts, categories and maybe the tags if it's not too difficult. But your database doesn't include the users, so it won't work as is. You will need to convert your multi site db, back to a clean DB. Other tables are ignored, the import plugin only looks for specific table names in the first place. The plugin would work fine for users with a largely modified forum. But it seems that a multi site import is another step entirely.

    If I'm not mistaken, I believe wp_users would be on the "master" install of Wordpress, then each subsidiary Wordpress site (wp_1_ , wp_2_ , etc) just contains the information for that wordpress. The user accounts would be verified against the "master database", which leads to an issue when you want to take one of those multi site installs, and import it into NodeBB. You would need to merge the master DB with wp_9_, in a way that didn't cause the loss of any posts, then use the importer.


  • Ok converting multi site back to normal wordpress is easy, that can be done.

    But as an ALTERNATE option, won't this plugin work if I exported the required site from a multi site to a normal wordpress installation using wordpress exporter and then go for nodebb export? I think it should work, because a fresh WordPress installation only contains the core tables. So once the users, posts and comments are transferred to a fresh wordpress installation, we can use the nodebb plugin for the final export.


  • @vegetagz6 said:

    Ok converting multi site back to normal wordpress is easy, that can be done.

    But as an ALTERNATE option, won't this plugin work if I exported the required site from a multi site to a normal wordpress installation using wordpress exporter and then go for nodebb export? I think it should work, because a fresh WordPress installation only contains the core tables. So once the users, posts and comments are transferred to a fresh wordpress installation, we can use the nodebb plugin for the final export.

    What I said and what you said were the same thing, so yes. If there's a tool that does it for you, then that would be fine. As long as the final database has all of the tables in this image and they're populated with the right number of entries:

    this one

  • Plugin & Theme Dev

    I don't even mind handling a "multi-site" WP DB, in one exporter, as long as everything is in a single Database, not multi (that would make it really hairy)

    I can add an advanced-custom-config that accepts a extra custom JSON for the exporter in the UI, then plumbs it through to the exporter module, so something like

    exporterConfig = {
      dbhost: 'localhost',
      dbname: 'wp'
      dbuser: 'user',
      dbpassword: 'password',
      tablePrefix: 'wp_',
      custom: {
        // anything you want here, 
        // that would be a result of a JSON.parse(/* what's on an advanced UI field */ )
      
        // in the case of the wp exporter,  a new custom option is introduced to specify which WP site you want to target
        // the UI value would be string '{ "wpsite": "1" }'
        wpsite: "1,2" // that means, site 1 and site 2,   
        // For each Site, the exporter would create a new NodeBB Parent-Category, that's doable too
        // wpsite: "all", // all sites, you guessed it
      }
    }
    

    however, my problem is still the same @vegetagz6,

    where are the users?

    where are the posts?

    is it possible that you might have another database?


  • @bentael I would try and find someone else with a standard WP site, as it's looking like there's at least 12 databases here, if the schema of database numbering is anything to go by.

  • Plugin & Theme Dev

    okay.png,

    @vegetagz6 unless you can figure out the answers to my questions, I don't think i can help you much, sorry 😞

    anyone else has a standard WP DB with some data?


  • Mentioning previous people that wanted this plugin

    @Flexi-Hackify , @Tan-Nguyen , @John-Smith , @Hawon-Nguyen


  • Ok am sending u a standard database, which has all the standard tables like wp_comments, wp_users etc. Check your pm and lemme know 🙂

    Ya the previous database was heavily modified, in the sense that it was part of a multisite installation, and belonged to one of the most active site I have.


  • @a_5mith any updates from Bentael on this plugin, yet?

  • Plugin & Theme Dev

    @vegetagz6 no updates yet - maybe next week, sorry ive been very busy combined with very sick for the past 2 weeks.

  • Plugin & Theme Dev

    wordpress exporter released,

    @vegetagz6
    I only tested against your "dota" db, looks good so far except that you only use 1 out of 2 categories. Your "droid" db had only 2 users..

    Needs more testing of course,

    but can't do much without many DB Dumps,

    I also need to export tags for topics but I need more dumps to test with.

  • Plugin & Theme Dev

    @John-Smith
    @Hawon-Nguyen
    @Flexi-Hackify
    @Tan-Nguyen

    read 1 post up
    To use it, you just need to install https://github.com/akhoury/nodebb-plugin-import and follow instructions in the Admin plugin page


  • @bentael Thank you very much for all of your hard works. What happen to the old post link. I mean will it automatically redirect to the new link (nodebb)?

  • Plugin & Theme Dev

    @Tan-Nguyen nothing redirects automatically, you have to set something in the middle to redirect. The import plugin will provide a map from old to new.

    Read this note: https://github.com/akhoury/nodebb-plugin-import#imported-now-what focus on the redirect section

Suggested Topics

  • 3
  • 9
  • 3
  • 21
  • 2
| |