There is a lot of settings 😉 I am using this my self. My settings is about 1 week
Upgrade "Refresh post-upload associations" in 1.9.x causes error
Dravere last edited by Dravere
@baris Sure, sadly it didn't fix the problem.
Is much slower, but threw the same error after about 2 minutes at around 4%.
Still slower but faster than previous one. Threw the same error after about 2 minutes at around 5%.
Huge performance improvement. Would probably finish in 25 instead of 50 minutes. But still threw the error after about 3 minutes at 14%
Edit: Will have to move this test environment tomorrow. I'll report back if I still can reproduce the error in the new test environment.
@baris Was able to try it out on my new test environment. This new environment has extremely better hardware specs. But it still crashed with the same error after about 2 minutes at 14%. This was with the default 1.9.2 code.
Could you try adding another option to the script here:
@Dravere Yes, I echo that... especially now that your migration went so smoothly
If you are worried we can provide our pub keys so the server remains under your control... but don't let us near the actual production box.
I can't send you the data. I think I would break several laws if I would do that at this point
Also I don't know how important that work is. I mean my production database is already on 1.9.2 since I migrated directly to that version from the old forum (phpbb2). So this really doesn't affect me at all. I just wanted to report it, perhaps try to understand it and learn something from it. Especially since my knowledge of MongoDB is still quite limited.
This error is happening on an earlier test migration to 1.8.2 that I have still around. So in theory I could just throw it away. Especially if you have no other reports with this problem.
I'll gladly try it with smaller batch sizes. But I kind of doubt that this will make a lot of difference. I have a batch process in my modified solr plugin but with a batch size of 1000 and it is working fine there (Yes, I'll work on the pull requests for solr, I know they are still open ).
@dravere can you make a change for me and run the upgrade script again?
console.log('uploads', pid, uploads);here
The error is caused by a timeout and the only place I can think of halting execution is that while loop. The timeout seems to be 10minutes and if
cursor.next()is not called for 10minutes you get that cursor not found error.
Commenting out the fast path (these three lines) seems to be working so far (although extremely slowly).
My instance would previously get killed at around 15% (there were a lot of posts from two forum softwares ago when pasting files into a forum wasn't a thing), but now it's at 90%. It's been about 4 hours since I started this attempt.
@Dravere ping, as this may be of interest to you
It is not a problem in NodeBB. It seems to be a problem in the mongodb package. Downgrading it to version 3.0.4 and then only executing
./nodebb upgrade -sso that the mongodb package isn't upgraded back to 3.0.8 makes the migration work. And it makes it work really fast. It took like 5 minutes.
Found it via this issue in mongoose: https://github.com/Automattic/mongoose/issues/6504
And the issue going along in JIRA: https://jira.mongodb.org/browse/NODE-1482
If someone has an account in this JIRA tracker, we perhaps should report there, that we are able to reproduce that bug.
I commented on the issue in jira and posted a link to this topic.
Well done, that's some thorough investigating @Dravere !