How can I backdate topics and posts (for migration purposes)?

Moved Developer FAQ
  • Hello, i'm moving my old forum to a Nodebb engine and got several issues.

    1. There is no possibility to set timestamp for a new topic. I need to save topic (and 1st post also) creation date and time.
    2. When i use timestamp for post replying, it doesn't work too.
      This timestamp is from 2011
    DEBU[0000] post body: {"cid":87,"title":"topic","content":"my first post","tags":null,"timestamp":1309975440000,"_uid":272362} 
    DEBU[0000] POST URL: https://forum.mydomain.ltd/api/v3/topics/ 
    INFO[0000] 200 OK  
    

    But i got this as response:

    INFO[0000] {Tid:2447, UID:272362, Cid:87, Title:"topic", Slug:"2447/topic", MainPid:2471, Postcount:1, Viewcount:0, Postercount:1, Scheduled:false, Deleted:0, DeleterUID:0, TitleRaw:"topic", Locked:0, Pinned:0, Timestamp:1676286542609, TimestampISO:"2023-02-13T11:09:02.609Z", Lastposttime:1676286542609, LastposttimeISO:"2023-02-13T11:09:02.609Z", PinExpiry:0, PinExpiryISO:"", Upvotes:0, Downvotes:0, Votes:0, TeaserPid:0, Thumbs:[]interface {}{}, NumThumbs:0, Category:struct {}{}, User:struct {}{}, Teaser:struct {}{}, Tags:[]interface {}{}, IsOwner:true, Ignored:false, Unread:false, Bookmark:1, Unreplied:true, Icons:[]interface {}{}, Thumb:"", Index:0, MainPost:map[string]interface {}{"bookmarked":false, "cid":87, "content":"<p dir=\"auto\">my first post</p>\n", "display_delete_tools":true, "display_edit_tools":true, "display_moderator_tools":true, "display_move_tools":true, "index":0, "ip":"192.168.1.17", "isMain":true, "pid":2471, "selfPost":false, "tid":2447, "timestamp":1.676286542609e+12, "timestampISO":"2023-02-13T11:09:02.609Z", "topic":map[string]interface {}{"cid":87, "mainPid":2471, "postcount":1, "scheduled":false, "slug":"2447/topic", "tid":2447, "timestamp":1.676286542609e+12, "timestampISO":"2023-02-13T11:09:02.609Z", "title":"topic", "titleRaw":"topic", "uid":272362}, "uid":272362, "user":map[string]interface {}{"banned":false, "banned:expire":0, "banned_until":0, "banned_until_readable":"Not Banned", "custom_profile_info":[]interface {}{}, "displayname":"Свуй", "groupTitle":"[\"Зергеры\"]", "groupTitleArray":[]interface {}{"Зергеры"}, "icon:bgColor":"#2196f3", "icon:text":"С", "lastonline":1.676286271926e+12, "lastonlineISO":"2023-02-13T11:04:31.926Z", "muted":false, "mutedUntil":0, "picture":interface {}(nil), "postcount":6, "reputation":4, "selectedGroups":[]interface {}{}, "signature":"", "status":"online", "topiccount":6, "uid":272362, "username":"Свуй", "userslug":"свуй"}, "votes":0}}}
    

    So the post has current timestamp. May be i do something wrong?
    At least, please add a possibility to set timestamp for new topics in

    POST /api/v3/topics/
    

    Thanks!

  • шЫкель грубыйШ шЫкель грубый marked this topic as a regular topic on
  • Hi @шЫкель-грубый unfortunately we cannot update the API to allow timestamps to be passed in at will because the API is used by the frontend for regular posting (that is the intended use-case). If we allowed it, then people would abuse the API and make posts with their own timestamps, backdating them at will.

    The only supported use case is setting a timestamp in the future, which will work if the calling user is an admin or has the requisite privilege for scheduling topics.

    You have two options:

    1. Use nodebb-plugin-import. This plugin uses internal methods that bypass sanity checks in the data, and let you migrate data in quicker.
    2. Comment out the requisite code guarding timestamp altering. For both topics and posts, I think you will need to comment out this line: https://github.com/NodeBB/NodeBB/blob/326b92687fa5d2b68cc5f55275c565a43bf6a16e/src/api/helpers.js#L16

    If doing #2, after you comment out the line, restart NodeBB and run your migration script, and topics and posts should be created without their timestamps overwritten.

  • @julian said in Timestamp issues:

    The only supported use case is setting a timestamp in the future, which will work if the calling user is an admin or has the requisite privilege for scheduling topics.

    This is not documented in the write API documentation, but I will amend that now.

    This is now documented in the write API documentation.

  • julianJ julian moved this topic from Technical Support on
  • This is a good question, I have updated the title and moved this to the Developer FAQ.

  • julianJ julian referenced this topic on
  • Thanks for fast response!
    I have admin privilege, but it doesn't help for now.
    My old forum is not a Nodebb installation, it's vBulletin. I've wrote a migration script which takes data from sql database and add topics and posts to Nodebb via api. If there is no way to change timestamps, looks like i'll change in in redis for all necessary resources. It will make my script much more complex but i need to complete the migration.
    I'll also try to comment date changing in sources. I'm not familiar with JS but it looks obviously enough.
    Thanks anyway

  • @шЫкель-грубый said in How can I backdate topics and posts (for migration purposes)?:

    I'll also try to comment date changing in sources. I'm not familiar with JS but it looks obviously enough.

    Yes, this is probably the most direct solution. It'll get your script working with the correct timestamps, all you have to do is comment out that line in the file, and restart NodeBB.


Suggested Topics


  • 4 Votes
    6 Posts
    322 Views

    @DownPW yeah for Windows, it's just the executable as downloaded from the GitHub releases page.

    The script for Linux and Mac does the same thing, it just automatically updates the $PATH variable for you.

  • Threaded Discussions

    Moved Solved Developer FAQ
  • 1 Votes
    2 Posts
    464 Views

    thank you for this.

  • 1 Votes
    1 Posts
    74 Views

    Your plugin may want to expose some user-specific options, and to accomplish that, you'll want to create a page accessible from within their user profile.

    In Harmony, plugin-added pages are added to the left-hand sidebar:
    ce07dc7d-e8f3-45a9-ac73-72f1f7309af2-image.png

    In Persona, plugin-added pages are behind the overflow menu:
    af0633f5-175d-45ff-802b-342a0885dd68-image.png

    You'll need to add listeners to two hooks, and modify your page template accordingly.

    static:app.load

    Live example

    You'll need to specify a route in the user profile using this hook. In it, note the accountMiddlewares block, which contains some common middlewares that are sensible defaults. You'll need middleware.buildAccountData in order to retrieve some boilerplate data that all account routes need.

    filter:user.profileMenu

    Live example

    You'll then want to specify the menu option, including label, icon, and visibility options. visibility allows you to specify which users can see the option (e.g. self only, admins only, etc.)

    The template

    In your template for the user page, you'll want to prefix it with <!-- IMPORT partials/account/header.tpl --> and suffix it with <!-- IMPORT partials/account/footer.tpl -->. These two lines will wrap your template content with the theme-specific structure. For example, in Harmony, a sidebar is used in the accounts pages. The header and footer partials will ensure they are also present in your template. The middleware.buildAccountData middleware you added in the first step will ensure the data necessary is present.

  • 0 Votes
    4 Posts
    200 Views

    Awesome! Thank you.