Migration line breaks



  • Hi,
    I've successfully installed and imported a bunch of topics to a fresh NodeBB.
    The software is great, I love the speed and simplicity compared to other forums!

    I do have a problem with the data imported. I'd assumed the "\n" text would be parsed as line breaks, but that's not picked up by the markdown so I get posts like
    " Hello all,\n\nI am in school and the prof has presented us with a question in relation to..."

    Is there anything I can use to convert these or is a database (MongoDB) find and replace the best option here?

    Many thanks,
    Chris



  • So the solution I have found is to replace all of the \n line breaks with %0A characters. The API then sends these to display correctly.

    I'm not sure if there is a better way, but that solves my problem once I purge the topics and repost.

    Thanks for all your help.



  • What is the raw post content in the database? You can get one by db.objects.findOne({_key: "post:1"})



  • Thanks, I've just added the mongo steps for anyone else like me who's not super technical.
    Log into shell
    $ mongo

    use nodebb

    switched to db nodebb

    db.auth({user: "nodebb", pwd: "YourPassword"})
    db.objects.findOne({_key: "post:7"})

    My output

        "_id" : ObjectId("5cbf2645e60ba5e377906c57"),
    
        "_key" : "post:7",
    
        "content" : "I work in a 15 person DBA shop that supports 425+ SQL servers, one-third of which are production. All 15 DBAs are sysadmin on all of our SQL Servers. We do not separate duties into dev, QA, and prod, and we are not developers or business people. \n\nWe don't often use the sa account, but there are rare occasions where windows authentication was not available on a server and we had to use the sa account to get in. \n\nWe have been looking at several solutions to the sa account problem in the context of a SOX audit. The problem as defined is that the sa account allows anonymous access using a priviledged, shared account. So we are trying to address the problem either by making the sa account inaccessible, or monitoring its usage. We have identified three solutions. \n\n1) Turn on All auditing and monitor for someone logging in as sa, and then try to match the login with someone's windows account. \n\n2) Put a trace on every production server and monitor for someone logging in a as sa, and then try to match the login with someone's windows account. \n\n3) Randomize the sa password daily, and modify the system stored procedure sp_password to record whose account attemted to change the sa password. Also monitor for the use of sp_configure to set Allow Updates to 1. \n\nHowever, these three solutions have serious drawbacks: \n\n1) auditing adds performance load to the system and it renders the errlog and the application eventlog useless because of all the noise. Also, you still don't get any information about which windows account actually logged in as sa. \n\n2) tracing adds performance load to the system, especially on very busy systems where the load increases exponentially as the system becomes busier. Also, you still don't get any information about which windows account actually logged in as sa. \n\n3) Modifying system stored procedures is not recommended by Microsoft PSS and could possibly render our support agreements with them void. The same goes for removing the sa account entirely. \n\n\nMicrosoft recommends going to Windows Only authentication mode. Great! We'd love to, but we aren't developers. We can't just flip a switch and then let the application support people sort out the mess. Ultimately, this is a difficulty that Microsoft created by having an sa account that had no controls on it at all. When I've asked MS what they are doing about this, they are barely aware of something called SOX. They've been useless. \n\n\nWith the number of servers we support, it would be impossible to operate effectively without DBAs being sysadmins. Impossible! We have good separation between developers, change management approvers, and DBAs. We have good processes and procedures, but we have been operating in the modern environment of competition with outsourcers and co-sourcers. If we can't operate efficiently, we may not be a support group worth keeping internal. \n\nI would most appreciate somebody sharing a real solution to this problem of SOX and the SQL Server sa account. Remember, we support a large enterprise of servers. We have to automate everything to be effective and efficient. Half-baked solutions won't work. \n\nThanks\n\nJimTuck",
    
        "deleted" : 0,
    
        "pid" : 7,
    
        "tid" : 4,
    
        "timestamp" : 1556031045624,
    
        "uid" : 1,
    
        "deleterUid" : 0,
    
        "downvotes" : 0,
    
        "edited" : 1556031442324,
    
        "editedISO" : "",
    
        "editor" : 1,
    
        "timestampISO" : "2019-04-23T14:50:45.624Z",
    
        "upvotes" : 0,
    
        "votes" : 0
    

    }



  • The new lines look fine, is the markdown plugin activated?



  • 2b602bd5-6fce-4bac-ba74-91877730a8cd-image.png

    Looks on to me?
    I've also done a rebuild just to be sure.



  • Yeah looks fine to me, is your forum public? Can you send a link to that post you shared?



  • Not the same post but the generic problem:
    https://www.sarbanes-oxley-forum.com/topic/1606/the-bane-of-my-life



  • Do you get proper linebreaks when you make a new post? Or is this happening on new posts as well?



  • https://www.sarbanes-oxley-forum.com/topic/2317/new-post

    Works if I press enter, but not if I type \n.
    [edit - but \n doesn't work here either]

    I posted these using the API and CURL - is it possible that there is a character set issue, or is mongoDB utf-8 by default?



  • So it looks like the API translates \n to \\n and the markup plugin doesn't pickup \n\n as a double new line.

    Can I send a different string to create a new line? or somehow prevent the API translation?
    Thanks



  • Did you use the write api plugin to post those? Maybe it's an issue with the plugin. @julianlam



  • @Chris1 Can you provide the curl request you used to reproduce the issue. Thanks.



  • Example:
    curl -H "Authorization: Bearer xxxxxxxxxxxxxxxxxxxxxx" --data "title=AeA Intern looking for information&content=I am an Intern for AeA and stumbled across this forum. \nI am looking for information on this, particularly information reguarding the issues that have come up reguarding Big 4 and others droping smaller buisnesses because they do not have the resources to handle all of these audits. I am wondering if anyone here can point me in the direction i need to go to find information on this. I am really focused on Small Buisness, but i also need to find if there is any plans to ammend the regulations to fix the problems. If anyone has ANY information, please reply or email me. thanks so much.&cid=17&_uid=1" http://localhost:4567/api/v1/topics

    Yes, using the API write plugin.

    Thanks



  • So the solution I have found is to replace all of the \n line breaks with %0A characters. The API then sends these to display correctly.

    I'm not sure if there is a better way, but that solves my problem once I purge the topics and repost.

    Thanks for all your help.


Log in to reply
 

Suggested Topics

  • 1
  • 3
  • 2
  • 5
  • 18
| |