Migration line breaks

Solved Technical Support
  • Hi,
    I've successfully installed and imported a bunch of topics to a fresh NodeBB.
    The software is great, I love the speed and simplicity compared to other forums!

    I do have a problem with the data imported. I'd assumed the "\n" text would be parsed as line breaks, but that's not picked up by the markdown so I get posts like
    " Hello all,\n\nI am in school and the prof has presented us with a question in relation to..."

    Is there anything I can use to convert these or is a database (MongoDB) find and replace the best option here?

    Many thanks,
    Chris

  • So the solution I have found is to replace all of the \n line breaks with %0A characters. The API then sends these to display correctly.

    I'm not sure if there is a better way, but that solves my problem once I purge the topics and repost.

    Thanks for all your help.

  • What is the raw post content in the database? You can get one by db.objects.findOne({_key: "post:1"})

  • Thanks, I've just added the mongo steps for anyone else like me who's not super technical.
    Log into shell
    $ mongo

    use nodebb

    switched to db nodebb

    db.auth({user: "nodebb", pwd: "YourPassword"})
    db.objects.findOne({_key: "post:7"})

    My output

        "_id" : ObjectId("5cbf2645e60ba5e377906c57"),
    
        "_key" : "post:7",
    
        "content" : "I work in a 15 person DBA shop that supports 425+ SQL servers, one-third of which are production. All 15 DBAs are sysadmin on all of our SQL Servers. We do not separate duties into dev, QA, and prod, and we are not developers or business people. \n\nWe don't often use the sa account, but there are rare occasions where windows authentication was not available on a server and we had to use the sa account to get in. \n\nWe have been looking at several solutions to the sa account problem in the context of a SOX audit. The problem as defined is that the sa account allows anonymous access using a priviledged, shared account. So we are trying to address the problem either by making the sa account inaccessible, or monitoring its usage. We have identified three solutions. \n\n1) Turn on All auditing and monitor for someone logging in as sa, and then try to match the login with someone's windows account. \n\n2) Put a trace on every production server and monitor for someone logging in a as sa, and then try to match the login with someone's windows account. \n\n3) Randomize the sa password daily, and modify the system stored procedure sp_password to record whose account attemted to change the sa password. Also monitor for the use of sp_configure to set Allow Updates to 1. \n\nHowever, these three solutions have serious drawbacks: \n\n1) auditing adds performance load to the system and it renders the errlog and the application eventlog useless because of all the noise. Also, you still don't get any information about which windows account actually logged in as sa. \n\n2) tracing adds performance load to the system, especially on very busy systems where the load increases exponentially as the system becomes busier. Also, you still don't get any information about which windows account actually logged in as sa. \n\n3) Modifying system stored procedures is not recommended by Microsoft PSS and could possibly render our support agreements with them void. The same goes for removing the sa account entirely. \n\n\nMicrosoft recommends going to Windows Only authentication mode. Great! We'd love to, but we aren't developers. We can't just flip a switch and then let the application support people sort out the mess. Ultimately, this is a difficulty that Microsoft created by having an sa account that had no controls on it at all. When I've asked MS what they are doing about this, they are barely aware of something called SOX. They've been useless. \n\n\nWith the number of servers we support, it would be impossible to operate effectively without DBAs being sysadmins. Impossible! We have good separation between developers, change management approvers, and DBAs. We have good processes and procedures, but we have been operating in the modern environment of competition with outsourcers and co-sourcers. If we can't operate efficiently, we may not be a support group worth keeping internal. \n\nI would most appreciate somebody sharing a real solution to this problem of SOX and the SQL Server sa account. Remember, we support a large enterprise of servers. We have to automate everything to be effective and efficient. Half-baked solutions won't work. \n\nThanks\n\nJimTuck",
    
        "deleted" : 0,
    
        "pid" : 7,
    
        "tid" : 4,
    
        "timestamp" : 1556031045624,
    
        "uid" : 1,
    
        "deleterUid" : 0,
    
        "downvotes" : 0,
    
        "edited" : 1556031442324,
    
        "editedISO" : "",
    
        "editor" : 1,
    
        "timestampISO" : "2019-04-23T14:50:45.624Z",
    
        "upvotes" : 0,
    
        "votes" : 0
    

    }

  • The new lines look fine, is the markdown plugin activated?

  • 2b602bd5-6fce-4bac-ba74-91877730a8cd-image.png

    Looks on to me?
    I've also done a rebuild just to be sure.

  • Yeah looks fine to me, is your forum public? Can you send a link to that post you shared?

  • Not the same post but the generic problem:
    https://www.sarbanes-oxley-forum.com/topic/1606/the-bane-of-my-life

  • Do you get proper linebreaks when you make a new post? Or is this happening on new posts as well?

  • https://www.sarbanes-oxley-forum.com/topic/2317/new-post

    Works if I press enter, but not if I type \n.
    [edit - but \n doesn't work here either]

    I posted these using the API and CURL - is it possible that there is a character set issue, or is mongoDB utf-8 by default?

  • So it looks like the API translates \n to \\n and the markup plugin doesn't pickup \n\n as a double new line.

    Can I send a different string to create a new line? or somehow prevent the API translation?
    Thanks

  • Did you use the write api plugin to post those? Maybe it's an issue with the plugin. @julianlam

  • @Chris1 Can you provide the curl request you used to reproduce the issue. Thanks.

  • Example:
    curl -H "Authorization: Bearer xxxxxxxxxxxxxxxxxxxxxx" --data "title=AeA Intern looking for information&content=I am an Intern for AeA and stumbled across this forum. \nI am looking for information on this, particularly information reguarding the issues that have come up reguarding Big 4 and others droping smaller buisnesses because they do not have the resources to handle all of these audits. I am wondering if anyone here can point me in the direction i need to go to find information on this. I am really focused on Small Buisness, but i also need to find if there is any plans to ammend the regulations to fix the problems. If anyone has ANY information, please reply or email me. thanks so much.&cid=17&_uid=1" http://localhost:4567/api/v1/topics

    Yes, using the API write plugin.

    Thanks

  • So the solution I have found is to replace all of the \n line breaks with %0A characters. The API then sends these to display correctly.

    I'm not sure if there is a better way, but that solves my problem once I purge the topics and repost.

    Thanks for all your help.


Suggested Topics


  • 0 Votes
    2 Posts
    46 Views

    @f10w said in Migrating to NodeBB from IP.Board 3.4.9:

    I am rather surprised to find no official guide on how to migrate from another software (e.g., Xenforo, IPS, vBulletin) to NodeBB.

    Unfortunately, one doesn't exist - see below

    Import data from IPBoard

    @julian thanks, @kamov emailed me, I'll update back here if I end up writing the IPBoard exporter.

    favicon

    NodeBB Community (community.nodebb.org)

  • 0 Votes
    12 Posts
    837 Views

    @JaredBusch said in How can I migrate from redis to postgresql/mongodb:

    Found a pull request for nodebb-plugin-import that gives it postgresql capability.

    Well, it seem that I cannot install v1.12.1 as required by this plugin.

    npm never successfully builds.

    [jbusch@obelisk nodebb]$ sudo -u nodebb ./nodebb setup package.json not found. Populating package.json... OK Dependencies outdated or not yet installed. Installing them now... npm notice npm notice New minor version of npm available! 8.3.1 -> 8.12.2 npm notice Changelog: https://github.com/npm/cli/releases/tag/v8.12.2 npm notice Run npm install -g [email protected] to update! npm notice npm ERR! code ERESOLVE npm ERR! ERESOLVE unable to resolve dependency tree npm ERR! npm ERR! While resolving: [email protected] npm ERR! Found: [email protected] npm ERR! node_modules/pg npm ERR! pg@"^7.4.0" from the root project npm ERR! npm ERR! Could not resolve dependency: npm ERR! peer pg@"^8" from [email protected] npm ERR! node_modules/pg-cursor npm ERR! pg-cursor@"^2.0.0" from the root project npm ERR! npm ERR! Fix the upstream dependency conflict, or retry npm ERR! this command with --force, or --legacy-peer-deps npm ERR! to accept an incorrect (and potentially broken) dependency resolution. npm ERR! npm ERR! See /home/nodebb/.npm/eresolve-report.txt for a full report. npm ERR! A complete log of this run can be found in: npm ERR! /home/nodebb/.npm/_logs/2022-06-18T01_31_52_584Z-debug-0.log Error installing dependencies! message: Command failed: npm install --production stdout: null stderr: null /opt/nodebb/src/cli/package-install.js:54 throw e; ^ Error: Command failed: npm install --production at checkExecSyncError (node:child_process:828:11) at Object.execSync (node:child_process:902:15) at Object.installAll (/opt/nodebb/src/cli/package-install.js:45:9) at Object.<anonymous> (/opt/nodebb/src/cli/index.js:60:18) at Module._compile (node:internal/modules/cjs/loader:1103:14) at Object.Module._extensions..js (node:internal/modules/cjs/loader:1155:10) at Module.load (node:internal/modules/cjs/loader:981:32) at Function.Module._load (node:internal/modules/cjs/loader:822:12) at Module.require (node:internal/modules/cjs/loader:1005:19) at require (node:internal/modules/cjs/helpers:102:18) { status: 1, signal: null, output: [ null, null, null ], pid: 1461, stdout: null, stderr: null } [jbusch@obelisk nodebb]$

    Built just fine to v2.2.0 though.

    [jbusch@obelisk nodebb]$ sudo -u nodebb ./nodebb setup Dependencies outdated or not yet installed. Installing them now... npm WARN deprecated [email protected]: this library is no longer supported npm WARN deprecated [email protected]: Multer 1.x is affected by CVE-2022-24434. This is fixed in v1.4.4-lts.1 which drops support for versions of Node.js before 6. Please upgrade to at least Node.js 6 and version 1.4.4-lts.1 of Multer. If you need support for older versions of Node.js, we are open to accepting patches that would fix the CVE on the main 1.x release line, whilst maintaining compatibility with Node.js 0.10. npm WARN deprecated [email protected]: request-promise-native has been deprecated because it extends the now deprecated request package, see https://github.com/request/request/issues/3142 npm WARN deprecated [email protected]: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See https://v8.dev/blog/math-random for details. npm WARN deprecated [email protected]: request has been deprecated, see https://github.com/request/request/issues/3142 npm WARN deprecated [email protected]: support for ECMAScript is superseded by `uglify-js` as of v3.13.0 npm WARN deprecated [email protected]: You can find the new Popper v2 at @popperjs/core, this package is dedicated to the legacy v1 added 922 packages, and audited 923 packages in 2m 76 packages are looking for funding run `npm fund` for details 5 high severity vulnerabilities Some issues need review, and may require choosing a different dependency. Run `npm audit` for details. OK 2022-06-18T01:40:58.599Z [1600] - info: NodeBB Setup Triggered via Command Line Welcome to NodeBB v2.2.0! This looks like a new installation, so you'll have to answer a few questions about your environment before we can proceed. Press enter to accept the default setting (shown in brackets).

    7d8a5a66-30f5-4d65-a5ec-ac395ecd0178-image.png

  • 0 Votes
    5 Posts
    457 Views

    @gotwf 9.5 (Ya I know is a very old version 😑 )

  • Migrate Nodebb

    Solved Technical Support
    0 Votes
    3 Posts
    2k Views

    @bri Thanks i have already moved my nodebb 🙂

  • 0 Votes
    2 Posts
    1k Views

    as temporary solution added to custom.css:

    .img-responsive {
    display:inline;
    }