Upgrade "Refresh post-upload associations" in 1.9.x causes error

Technical Support
  • GNU/Linux
    #7

    Could you try adding another option to the script here: batch: 50?

    25?

  • NodeBB
    #8

    @dravere If you don't mind you can send us a copy of your db as well so we can test it, maybe it has something to do with the data. You can send it to [email protected].

  • GNU/Linux
    #9

    @Dravere Yes, I echo that... especially now that your migration went so smoothly 😉

    If you are worried we can provide our pub keys so the server remains under your control... but don't let us near the actual production box. 😛

  • #10

    I can't send you the data. I think I would break several laws if I would do that at this point 😉

    Also I don't know how important that work is. I mean my production database is already on 1.9.2 since I migrated directly to that version from the old forum (phpbb2). So this really doesn't affect me at all. I just wanted to report it, perhaps try to understand it and learn something from it. Especially since my knowledge of MongoDB is still quite limited.

    This error is happening on an earlier test migration to 1.8.2 that I have still around. So in theory I could just throw it away. Especially if you have no other reports with this problem.

    I'll gladly try it with smaller batch sizes. But I kind of doubt that this will make a lot of difference. I have a batch process in my modified solr plugin but with a batch size of 1000 and it is working fine there (Yes, I'll work on the pull requests for solr, I know they are still open 😅).

  • NodeBB
    #11

    @dravere can you make a change for me and run the upgrade script again?

    Add console.log('uploads', pid, uploads); here

    The error is caused by a timeout and the only place I can think of halting execution is that while loop. The timeout seems to be 10minutes and if cursor.next() is not called for 10minutes you get that cursor not found error.

  • Gamers
    #12

    Commenting out the fast path (these three lines) seems to be working so far (although extremely slowly).

    My instance would previously get killed at around 15% (there were a lot of posts from two forum softwares ago when pasting files into a forum wasn't a thing), but now it's at 90%. It's been about 4 hours since I started this attempt.

  • GNU/Linux
    #13

    @Dravere ping, as this may be of interest to you

  • #14

    @baris @Ben-Lubar @julian Found the problem. The solution Ben describes works, but it is painfully slow. And also made no sense. So I continued searching and found it.

    It is not a problem in NodeBB. It seems to be a problem in the mongodb package. Downgrading it to version 3.0.4 and then only executing ./nodebb upgrade -s so that the mongodb package isn't upgraded back to 3.0.8 makes the migration work. And it makes it work really fast. It took like 5 minutes.

    Found it via this issue in mongoose: https://github.com/Automattic/mongoose/issues/6504
    And the issue going along in JIRA: https://jira.mongodb.org/browse/NODE-1482

    If someone has an account in this JIRA tracker, we perhaps should report there, that we are able to reproduce that bug.

  • NodeBB
    #15

    I commented on the issue in jira and posted a link to this topic.

  • GNU/Linux
    #16

    👏 Well done, that's some thorough investigating @Dravere !


Suggested Topics


  • number 1 at the end of the link

    Unsolved Technical Support
    0 Votes
    4 Posts
    64 Views

  • 0 Votes
    4 Posts
    261 Views

  • 1 Votes
    7 Posts
    1289 Views

  • 0 Votes
    3 Posts
    2162 Views

  • 0 Votes
    8 Posts
    1951 Views

| | | |