Source forums exporters to NodeBB, I need your DB Dump.


  • @bentael Okay. Let me see what I can do to scrub them.


  • @bentael You've got mail - it may be too large, let me know if it doesn't show.

  • Plugin & Theme Dev

    @Shard got it ! thanks,

    I will take a look at this the following weekend, I'm on a tight schedule with a long backlog of work.

  • Plugin & Theme Dev

    maybe sooner, looked at the schema, seems pretty straight forward 🙂

  • GNU/Linux Admin

    I didn't see the earlier post about post bookmarks until now, but that cannot be migrated over. Post bookmarks are specifically a client-side feature, meaning your bookmarks on one machine (e.g. a desktop) don't carry over to another (e.g. a laptop).

    It could be a server-backed feature, however, just at the current time, it is not.

  • Plugin & Theme Dev

    @julian are you talking about migrating old forums URLs that use the location.hash to "scroll to posts"?

    I don't think it would be too hard.

    https://github.com/akhoury/RedirectBB redirects solely on the server at the moment, however, i can have it
    serve a blank page (or with a "redirecting" text in it) and in that page a script tag that reads the hash value, the old _pid, sends a requests to RedirectBB instance to get the corresponding new pid , then re-write the URL with client-side javascript instead of doing it all on the server.

    Sure it won't be seamless, but, it shouldnt be too slow. The RediectBB instance loads the Map in memory, so there is no DB call.


  • thanks @bentael you rock man, glad my db was able to help unlock some of the import mysteries.

    setting up a test node now to screw around.


  • Clean nodebb install on digital ocean.. added the plugin, when i activated it now I can't get nodebb to stay up.. but we all know my history or breaking things here 🙂 going to screw around a bit and see if i can figure out what went bonkers here.

    so close...... I CAN TASTE IT!! 🙂

    edit: ok i used a different set of setup instructions let me kill this droplet and redo it quick


  • Okay.. when I actually go to Install the -vbulletin plugin via the ACP it kills the site and I can no longer connect to it.

    I'll email you admin/login details so you can poke around at what I broke in case it helps anyone else, I'm in no rush at all here just playing around..


  • Thanks for the help @bentael it's connected to the DB and importing now.. I'll post an update / screenshots / whatever once I get it to the promised land 🙂

  • Plugin & Theme Dev

    @steve don't forget to use the post-import tools to convert the content to Markdown,


  • I'm doing that now but it has been running well over an hour now.. Starting to wonder if it croaked on me or not.

    yup totally crapped out on me and killed the site again.. hey i'm a professional at this 🙂


  • here's the error I'm getting from the log file in ./nodebb log upon trying to start the site.

    info: [app] Shutdown (SIGTERM/SIGINT) Initialised.
    info: [app] Database connection closed.
    info: [app] Shutdown complete.


  • @Steve said:

    here's the error I'm getting from the log file in ./nodebb log upon trying to start the site.

    info: [app] Shutdown (SIGTERM/SIGINT) Initialised.
    info: [app] Database connection closed.
    info: [app] Shutdown complete.

    You will need to run ./nodebb dev, there will be a full error before that when it crashes, but loading the log after the error won't load it sometimes.

  • Plugin & Theme Dev

    @Steve

    • which exact version of NodeBB are u running against? (don't use master yet, use 0.5.0 or 0.5.0-4 if you have to)
    • also, check the "log on client" option, and paste the last few (5 maybe) logs you see on the browser
    • try running nodebb via either the ./nodebb dev or NODE_ENV=development node app (the latter skips using loader), paste the logs.
    • how big is your DB? is it the one you sent me? I was able to import it in few minutes.

  • Here's the full error log:

    Launching NodeBB in "development" mode.
    To run the production build of NodeBB, please use "forever".
    More Information: https://docs.nodebb.org/en/latest/running/index.html
    info: NodeBB v0.5.0 Copyright (C) 2013-2014 NodeBB Inc.
    info: This program comes with ABSOLUTELY NO WARRANTY.
    info: This is free software, and you are welcome to redistribute it under certain conditions.
    info:
    info: Time: Mon Sep 22 2014 09:42:02 GMT-0400 (EDT)
    info: Initializing NodeBB v0.5.0
    info: * using configuration stored in: /root/mynodebb/config.json
    info: * using redis store at 127.0.0.1:6379
    info: * using themes stored in: /root/mynodebb/node_modules
    info: Base Configuration OK.
    warn: You have no redis password setup!
    morgan deprecated morgan(options): use morgan("default", options) instead src/logger.js:128:22
    morgan deprecated default format: use combined format src/logger.js:128:22
    info: [notifications.init] Registering jobs.
    info: [user.startJobs] Registering User Jobs
    info: Enabling 'trust proxy'
    info: [plugins] Initializing plugins system
    info: [themes] Using theme nodebb-theme-lavender
    info: Middlewares loaded.
    info: [plugins] Found 1 LESS file(s) for plugin nodebb-plugin-markdown
    info: [plugins] Found 1 LESS file(s) for plugin nodebb-plugin-mentions
    info: [plugins] Found 2 js file(s) for plugin nodebb-plugin-mentions
    info: [plugins] Found 1 CSS file(s) for plugin nodebb-widget-essentials
    info: [plugins] Found 3 js file(s) for plugin nodebb-theme-lavender
    info: [plugins] Loaded plugin: nodebb-plugin-markdown
    info: [plugins] Loaded plugin: nodebb-plugin-soundpack-default
    info: [plugins] Loaded plugin: nodebb-plugin-mentions
    info: [plugins] Loaded plugin: nodebb-widget-essentials
    error: ENOENT, no such file or directory '../package.json'
    Error: ENOENT, no such file or directory '../package.json'
    at Object.fs.openSync (fs.js:438:18)
    at Object.fs.readFileSync (fs.js:289:15)
    at Object.me.readFileSync (/root/mynodebb/node_modules/nodebb-plugin-import-vbulletin/node_modules/fs-extra/node_modules/jsonfile/lib/jsonfile.js:31:26)
    at Object.<anonymous> (/root/mynodebb/node_modules/nodebb-plugin-import-vbulletin/admin/index.js:5:11)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Module.require (module.js:364:17)
    at require (module.js:380:17)
    at /root/mynodebb/src/plugins.js:153:45
    info: [app] Shutdown (SIGTERM/SIGINT) Initialised.
    info: [app] Database connection closed.
    info: [app] Shutdown complete.
    info: NodeBB v0.5.0 Copyright (C) 2013-2014 NodeBB Inc.
    info: This program comes with ABSOLUTELY NO WARRANTY.
    info: This is free software, and you are welcome to redistribute it under certain conditions.
    info:
    info: Time: Mon Sep 22 2014 09:42:03 GMT-0400 (EDT)
    info: Initializing NodeBB v0.5.0
    info: * using configuration stored in: /root/mynodebb/config.json
    info: * using redis store at 127.0.0.1:6379
    info: * using themes stored in: /root/mynodebb/node_modules
    info: Base Configuration OK.
    warn: You have no redis password setup!
    morgan deprecated morgan(options): use morgan("default", options) instead src/logger.js:128:22
    morgan deprecated default format: use combined format src/logger.js:128:22
    info: [notifications.init] Registering jobs.
    info: [user.startJobs] Registering User Jobs
    info: Enabling 'trust proxy'
    info: [plugins] Initializing plugins system
    info: [themes] Using theme nodebb-theme-lavender
    info: Middlewares loaded.
    info: [plugins] Found 1 LESS file(s) for plugin nodebb-plugin-markdown
    info: [plugins] Found 1 LESS file(s) for plugin nodebb-plugin-mentions
    info: [plugins] Found 2 js file(s) for plugin nodebb-plugin-mentions
    info: [plugins] Found 1 CSS file(s) for plugin nodebb-widget-essentials
    info: [plugins] Found 3 js file(s) for plugin nodebb-theme-lavender
    info: [plugins] Loaded plugin: nodebb-plugin-markdown
    info: [plugins] Loaded plugin: nodebb-plugin-soundpack-default
    info: [plugins] Loaded plugin: nodebb-plugin-mentions
    info: [plugins] Loaded plugin: nodebb-widget-essentials
    error: ENOENT, no such file or directory '../package.json'
    Error: ENOENT, no such file or directory '../package.json'
    at Object.fs.openSync (fs.js:438:18)
    at Object.fs.readFileSync (fs.js:289:15)
    at Object.me.readFileSync (/root/mynodebb/node_modules/nodebb-plugin-import-vbulletin/node_modules/fs-extra/node_modules/jsonfile/lib/jsonfile.js:31:26)
    at Object.<anonymous> (/root/mynodebb/node_modules/nodebb-plugin-import-vbulletin/admin/index.js:5:11)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Module.require (module.js:364:17)
    at require (module.js:380:17)
    at /root/mynodebb/src/plugins.js:153:45
    info: [app] Shutdown (SIGTERM/SIGINT) Initialised.
    info: [app] Database connection closed.
    info: [app] Shutdown complete.
    info: NodeBB v0.5.0 Copyright (C) 2013-2014 NodeBB Inc.
    info: This program comes with ABSOLUTELY NO WARRANTY.
    info: This is free software, and you are welcome to redistribute it under certain conditions.
    info:
    info: Time: Mon Sep 22 2014 09:42:04 GMT-0400 (EDT)
    info: Initializing NodeBB v0.5.0
    info: * using configuration stored in: /root/mynodebb/config.json
    info: * using redis store at 127.0.0.1:6379
    info: * using themes stored in: /root/mynodebb/node_modules
    info: Base Configuration OK.
    warn: You have no redis password setup!
    morgan deprecated morgan(options): use morgan("default", options) instead src/logger.js:128:22
    morgan deprecated default format: use combined format src/logger.js:128:22
    info: [notifications.init] Registering jobs.
    info: [user.startJobs] Registering User Jobs
    info: Enabling 'trust proxy'
    info: [plugins] Initializing plugins system
    info: [themes] Using theme nodebb-theme-lavender
    info: Middlewares loaded.
    info: [plugins] Found 1 LESS file(s) for plugin nodebb-plugin-markdown
    info: [plugins] Found 1 LESS file(s) for plugin nodebb-plugin-mentions
    info: [plugins] Found 2 js file(s) for plugin nodebb-plugin-mentions
    info: [plugins] Found 1 CSS file(s) for plugin nodebb-widget-essentials
    info: [plugins] Found 3 js file(s) for plugin nodebb-theme-lavender
    info: [plugins] Loaded plugin: nodebb-plugin-markdown
    info: [plugins] Loaded plugin: nodebb-plugin-soundpack-default
    info: [plugins] Loaded plugin: nodebb-plugin-mentions
    info: [plugins] Loaded plugin: nodebb-widget-essentials
    error: ENOENT, no such file or directory '../package.json'
    Error: ENOENT, no such file or directory '../package.json'
    at Object.fs.openSync (fs.js:438:18)
    at Object.fs.readFileSync (fs.js:289:15)
    at Object.me.readFileSync (/root/mynodebb/node_modules/nodebb-plugin-import-vbulletin/node_modules/fs-extra/node_modules/jsonfile/lib/jsonfile.js:31:26)
    at Object.<anonymous> (/root/mynodebb/node_modules/nodebb-plugin-import-vbulletin/admin/index.js:5:11)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Module.require (module.js:364:17)
    at require (module.js:380:17)
    at /root/mynodebb/src/plugins.js:153:45
    info: [app] Shutdown (SIGTERM/SIGINT) Initialised.
    info: [app] Database connection closed.
    info: [app] Shutdown complete.
    info: NodeBB v0.5.0 Copyright (C) 2013-2014 NodeBB Inc.
    info: This program comes with ABSOLUTELY NO WARRANTY.
    info: This is free software, and you are welcome to redistribute it under certain conditions.
    info:
    info: Time: Mon Sep 22 2014 09:42:05 GMT-0400 (EDT)
    info: Initializing NodeBB v0.5.0
    info: * using configuration stored in: /root/mynodebb/config.json
    info: * using redis store at 127.0.0.1:6379
    info: * using themes stored in: /root/mynodebb/node_modules
    info: Base Configuration OK.
    warn: You have no redis password setup!
    morgan deprecated morgan(options): use morgan("default", options) instead src/logger.js:128:22
    morgan deprecated default format: use combined format src/logger.js:128:22
    info: [notifications.init] Registering jobs.
    info: [user.startJobs] Registering User Jobs
    info: Enabling 'trust proxy'
    info: [plugins] Initializing plugins system
    info: [themes] Using theme nodebb-theme-lavender
    info: Middlewares loaded.
    info: [plugins] Found 1 LESS file(s) for plugin nodebb-plugin-markdown
    info: [plugins] Found 1 LESS file(s) for plugin nodebb-plugin-mentions
    info: [plugins] Found 2 js file(s) for plugin nodebb-plugin-mentions
    info: [plugins] Found 1 CSS file(s) for plugin nodebb-widget-essentials
    info: [plugins] Found 3 js file(s) for plugin nodebb-theme-lavender
    info: [plugins] Loaded plugin: nodebb-plugin-markdown
    info: [plugins] Loaded plugin: nodebb-plugin-soundpack-default
    info: [plugins] Loaded plugin: nodebb-plugin-mentions
    info: [plugins] Loaded plugin: nodebb-widget-essentials
    error: ENOENT, no such file or directory '../package.json'
    Error: ENOENT, no such file or directory '../package.json'
    at Object.fs.openSync (fs.js:438:18)
    at Object.fs.readFileSync (fs.js:289:15)
    at Object.me.readFileSync (/root/mynodebb/node_modules/nodebb-plugin-import-vbulletin/node_modules/fs-extra/node_modules/jsonfile/lib/jsonfile.js:31:26)
    at Object.<anonymous> (/root/mynodebb/node_modules/nodebb-plugin-import-vbulletin/admin/index.js:5:11)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Module.require (module.js:364:17)
    at require (module.js:380:17)
    at /root/mynodebb/src/plugins.js:153:45
    info: [app] Shutdown (SIGTERM/SIGINT) Initialised.
    info: [app] Database connection closed.
    info: [app] Shutdown complete.

    [loader] Experienced three start attempts in 10 seconds, most likely an error on startup. Halting.


  • @bentael said:

    @Steve

    • which exact version of NodeBB are u running against? (don't use master yet, use 0.5.0 or 0.5.0-4 if you have to)
    • also, check the "log on client" option, and paste the last few (5 maybe) logs you see on the browser
    • try running nodebb via either the ./nodebb dev or NODE_ENV=development node app (the latter skips using loader), paste the logs.
    • how big is your DB? is it the one you sent me? I was able to import it in few minutes.

    0.5.0 version and it's the same DB I sent you plus whatever content they created the past few days so a very small change in size.

    I can't check any options because I can't get into the site at all. I ran the parser after initial connect to database with plugin and got to this point.

  • Plugin & Theme Dev

    @Steve

    I'll test tonight, but you should be able to turn the site back on, NODE_ENV=development node app and see the error.

    if the ssh logins are still the same, I'll check out whats going on your end as well.


  • @bentael said:

    @Steve

    I'll test tonight, but you should be able to turn the site back on, NODE_ENV=development node app and see the error.

    if the ssh logins are still the same, I'll check out whats going on your end as well.

    ssh still the same, and no it still won't go on even with the error 🙂

    I have a feeling it never connected properly to the importer somewhere along the line. It's easy to just wipe and restart but I rather leave it like this to find a solution to keep on record in case it happens to anyone else.

  • Plugin & Theme Dev

    @Steve, so couple of things:

    • You re-installed nodebb-plugin-import-vbulletin instead of nodebb-plugin-import, that's why it was crashing nodebb on startup, but I pushed a fix to keep it from crashing, but that doesn't solve your problem, u need to install the latter plugin.
    • I've had a couple of problems importing your DB to nodebb@0.5.0, but after a couple of tryouts it worked (on my home machine, because I could not figure out your admin passwd); I've hit a couple of Bus error: 10 and a couple of Segmentation fault: 11, looks like that was happening when purging categories from a previous import. Also I hit few TypeErrors in NodeBB core, I am going to investigate these a little more and submit a Pull Request to the NodeBB core source. track issue here issue#51
    • I also found it necessary to change some redis configurations during the import, see this note

    Bottom line, it doesn't look it's 100% ready for large databases yet, sorry about that. If you want to migrate soon, I can personally help you with that, but you need to set a migration date and time, so you can freeze your current site and i can get the migration done meanwhile. It should not take me more than 15-30 minutes (if you're using redis and you have a ready server for that) - talk to me offline.

Suggested Topics

  • 6
  • 6
  • 4
  • 32
  • 2
| |