Frontend freeze on version later than v1.0.2

  • Swedes

    I am now running with v1.0.2 and updated plugins and it works. But if I try to upgrade to >v1.0.2 the frontend crashes with NO error, Just hangs. The fronted just stops taking in connections and it works to stop and restart nodebb. The crash comes after a while. 5-60 mins 😞

    Maybe som plugin setting that fucks the db up!
    Its not only one site. I have atleast 2> sitest that behaves this way.
    Tried node debug and nodebb debug, no success. Its just not f***ng working 😛


  • @Jenkler 1.0.3 seems ok to me, and if a clean install behaves ok, it seems something with the particular data or modifications of your install. That said its very rare that the process crahs without any log. Did you at least find when the crash happens in terms of front end actions you do that makes the crash happens?

    I would delete all info of plugins and custom data of BBDD and go progressively adding info to the BBDD in order to identify if it is a problem generated by the data.

  • Plugin & Theme Dev

    Looks like we got another case like this: https://community.nodebb.org/topic/8904/why-do-i-need-to-restart-my-nodebb-every-couple-of-hours

    Sadly I still can't give any further suggestions... No idea what might be wrong; As @jarey mentioned it most likely is a problem of database inconsistency. I guess only the @administrators are aware of what changes happened lately to the database.

  • GNU/Linux Admin

    @Jenkler Apologies if this was asked before, but have you tried running ./nodebb reset -p and seeing if the crashes still occur? If not, re-introduce plugins one-by-one until you find the culprit...

  • Community Rep

    I've started seeing this behavior as well on my dev environment, maybe it's related?

    Running 1.x.x branch latest as of May 20ish (1.0.3 @ a196c834d9aab4e1c27965dcca623ce12e61b4ae), mongo 3.0.7

    I'll be browsing around, and can see about 10 connections in total opened up in mongo, and then after some variable number of minutes, the application just becomes unresponsive; no error logs from node bb or anything. However, I see logs from mongo like this (note the timestamps):

    2016-05-29T21:38:52.811-0700 I ACCESS   [conn11] Successfully authenticated as principal nodebb on nodebb
    
    // then, after about 19 minutes:
    2016-05-29T21:57:19.778-0700 I NETWORK  [conn2] end connection 127.0.0.1:52223 (9 connections now open)
    2016-05-29T21:57:19.778-0700 I NETWORK  [conn7] end connection 127.0.0.1:52234 (8 connections now open)
    2016-05-29T21:57:19.779-0700 I NETWORK  [conn9] end connection 127.0.0.1:52236 (7 connections now open)
    2016-05-29T21:57:19.779-0700 I NETWORK  [conn4] end connection 127.0.0.1:52231 (6 connections now open)
    2016-05-29T21:57:19.779-0700 I NETWORK  [conn8] end connection 127.0.0.1:52235 (5 connections now open)
    2016-05-29T21:57:19.782-0700 I NETWORK  [conn6] end connection 127.0.0.1:52233 (4 connections now open)
    2016-05-29T21:57:23.474-0700 I NETWORK  [conn11] end connection 127.0.0.1:52251 (3 connections now open)
    2016-05-29T21:57:23.474-0700 I NETWORK  [conn3] end connection 127.0.0.1:52224 (3 connections now open)
    2016-05-29T21:57:23.474-0700 I NETWORK  [conn5] end connection 127.0.0.1:52232 (1 connection now open)
    2016-05-29T21:57:28.231-0700 I NETWORK  [conn10] end connection 127.0.0.1:52237 (0 connections now open)
    

    @Jenkler have you found mongo logs like this at all?

    I can't think of any plugins that I have installed that would cause this, the only thing I've been changing is the theme and the composer plugin, and some fresh npm installs... I haven't been switching the NBB branch at all.

  • Community Rep

    It seems pretty random, but here's another sample:

    2016-05-29T22:58:13.486-0700 I NETWORK  [initandlisten] connection accepted from 127.0.0.1:53521 #31 (1 connection now open)
    2016-05-29T22:58:13.561-0700 I ACCESS   [conn31] Successfully authenticated as principal nodebb on nodebb
    2016-05-29T23:02:00.357-0700 I NETWORK  [conn31] end connection 127.0.0.1:53521 (0 connections now open)
    2016-05-29T23:02:01.362-0700 I NETWORK  [initandlisten] connection accepted from 127.0.0.1:53603 #32 (1 connection now open)
    2016-05-29T23:02:01.440-0700 I ACCESS   [conn32] Successfully authenticated as principal nodebb on nodebb
    2016-05-29T23:04:03.492-0700 I NETWORK  [initandlisten] connection accepted from 127.0.0.1:53632 #33 (2 connections now open)
    2016-05-29T23:04:03.518-0700 I NETWORK  [initandlisten] connection accepted from 127.0.0.1:53634 #34 (3 connections now open)
    2016-05-29T23:04:23.842-0700 I NETWORK  [initandlisten] connection accepted from 127.0.0.1:53638 #35 (4 connections now open)
    2016-05-29T23:04:23.846-0700 I NETWORK  [initandlisten] connection accepted from 127.0.0.1:53639 #36 (5 connections now open)
    2016-05-29T23:04:23.846-0700 I NETWORK  [initandlisten] connection accepted from 127.0.0.1:53640 #37 (6 connections now open)
    2016-05-29T23:04:23.846-0700 I NETWORK  [initandlisten] connection accepted from 127.0.0.1:53641 #38 (7 connections now open)
    2016-05-29T23:04:23.848-0700 I NETWORK  [initandlisten] connection accepted from 127.0.0.1:53642 #39 (8 connections now open)
    2016-05-29T23:04:23.849-0700 I NETWORK  [initandlisten] connection accepted from 127.0.0.1:53643 #40 (9 connections now open)
    2016-05-29T23:04:23.850-0700 I NETWORK  [initandlisten] connection accepted from 127.0.0.1:53644 #41 (10 connections now open)
    2016-05-29T23:06:03.511-0700 I NETWORK  [conn33] end connection 127.0.0.1:53632 (9 connections now open)
    2016-05-29T23:06:03.519-0700 I NETWORK  [conn34] end connection 127.0.0.1:53634 (8 connections now open)
    2016-05-29T23:06:23.844-0700 I NETWORK  [conn35] end connection 127.0.0.1:53638 (7 connections now open)
    2016-05-29T23:06:23.844-0700 I NETWORK  [conn36] end connection 127.0.0.1:53639 (7 connections now open)
    2016-05-29T23:06:23.844-0700 I NETWORK  [conn37] end connection 127.0.0.1:53640 (5 connections now open)
    2016-05-29T23:06:23.844-0700 I NETWORK  [conn38] end connection 127.0.0.1:53641 (4 connections now open)
    2016-05-29T23:06:23.844-0700 I NETWORK  [conn39] end connection 127.0.0.1:53642 (3 connections now open)
    2016-05-29T23:06:23.847-0700 I NETWORK  [conn40] end connection 127.0.0.1:53643 (2 connections now open)
    2016-05-29T23:06:23.847-0700 I NETWORK  [conn41] end connection 127.0.0.1:53644 (1 connection now open)
    2016-05-29T23:07:06.570-0700 I NETWORK  [conn32] end connection 127.0.0.1:53603 (0 connections now open)
    
  • Swedes

    @julian No, that one i can try. I have now reverted all my dbs but i can try to setup the same stuff aging with v1.0.3 and my db.

    1. I will try the ./nodebb reset -p command
    2. I will check my mongodb logs!

    It would be nice to solve this 🙂

    Edit1: Dev setup with my db and v1.x.x is now done. Will wait for first crash. Also looking in mongodb.log 😉
    Edit2: Don know why but i get 4 auth instead of 1 when the site auths to the db. Normal ? My v1.0.2 sites only get 1 every time 🙂
    2016-05-30T11:53:10.080+0200 [conn75] authenticate db: dev_nodebb_se { authenticate: 1, user: "dev_nodebb_se", nonce: "xxx", key: "xxx" }
    2016-05-30T11:53:10.085+0200 [conn76] authenticate db: dev_nodebb_se { authenticate: 1, user: "dev_nodebb_se", nonce: "xxx", key: "xxx" }
    2016-05-30T11:53:10.087+0200 [conn77] authenticate db: dev_nodebb_se { authenticate: 1, user: "dev_nodebb_se", nonce: "xxx", key: "xxx" }
    2016-05-30T11:55:13.907+0200 [conn78] authenticate db: dev_nodebb_se { authenticate: 1, user: "dev_nodebb_se", nonce: "xxx", key: "xxx" }
    Edit3: First crash, nothing in mongodblogs and this in nodebb dev
    30/5 11:51 [68] - warn: [plugins] Expected method for hook 'filter:admin.header.build' in plugin 'nodebb-plugin-youtube-lite' not found, skipping.
    30/5 11:52 [68] - verbose: [translator] No resource file found for sv/markdown, using provided fallback language file
    30/5 11:52 [68] - warn: Route requested but not found: /vendor/jquery/timeago/locales/jquery.timeago.sv-short.js?_=1464602028233
    info: Imgbed: cache miss
    info: Imgbed: cache miss
    info: Imgbed: cache miss
    info: Imgbed: cache miss
    info: Imgbed: cache miss
    info: Imgbed: cache miss
    ...
    Edit4 Running ./nodebb reset -p and start nodebb dev again
    Edit5 Crash again. No errors no nothing Running ./nodebb reset -a and start nodebb dev again
    Edit6 To many db connections if you ask me 😉 Something seams wrong here!
    2016-05-30T12:20:34.571+0200 [conn115] authenticate db: dev_nodebb_se { authenticate: 1, user: "dev_nodebb_se", nonce: "xxx", key: "xxx" }
    2016-05-30T12:20:34.575+0200 [conn116] authenticate db: dev_nodebb_se { authenticate: 1, user: "dev_nodebb_se", nonce: "xxx", key: "xxx" }
    2016-05-30T12:20:34.579+0200 [conn117] authenticate db: dev_nodebb_se { authenticate: 1, user: "dev_nodebb_se", nonce: "xxx", key: "xxx" }
    2016-05-30T12:20:34.589+0200 [conn118] authenticate db: dev_nodebb_se { authenticate: 1, user: "dev_nodebb_se", nonce: "xxx", key: "xxx" }
    2016-05-30T12:20:34.592+0200 [conn119] authenticate db: dev_nodebb_se { authenticate: 1, user: "dev_nodebb_se", nonce: "xxx", key: "xxx" }
    2016-05-30T12:20:34.634+0200 [conn120] authenticate db: dev_nodebb_se { authenticate: 1, user: "dev_nodebb_se", nonce: "xxx", key: "xxx" }
    2016-05-30T12:20:38.577+0200 [conn121] authenticate db: dev_nodebb_se { authenticate: 1, user: "dev_nodebb_se", nonce: "xxx", key: "xxx" }
    Edit: Crash again and nothing more to test. Downgrading to v1.0.2 and see if it works again 🙂
    Edit: If I downgrade to v1.0.2 and restore the plugins nodebb crashes. This mean that the upgrade over v1.0.2 fucks your DB in some way. Now i have a broken v1.0.2. Not that my original v1.0.2 works fine on the same system. This is a major BUG!

    Something is wrong, but i cant find out what! More stuff i can try?


  • I'm having the same problem as well.

    I have checked all the things people has mentioned in this topic and several ones.

    1. Disabling plug-in
    2. Check memory (I have 1GB and 2GB of swap memory)
    3. no errors in nodebb logs and mongodb logs.

    Anything I tried is not working.

    If the problem is with version 1.0.3, is there a way to downgrade to 1.0.2 without loosing data?

    Thanks.

  • GNU/Linux Admin

    Are any of you up for running git-bisect to determine which commit started causing these issues?

    Because it only seems to occur on your instances, we cannot reproduce it and find a fix 😦

    We're running master branch here, with no issues.

    That said, do these errors occur on master?

  • Swedes

    Yes, same issue at master!


  • @julian I have installed nodebb less than a couple of weeks ago. I got master branch. Perhaps this only happens with new installations?

    At this point the forums is useless 😞 sometimes it freezes after half an hour, sometimes it runs no problem for a few hours or a whole night.

    If I somehow can help you diagnose the problem, tell me what I can do.

  • Swedes

    I am updating my system right now and wondering what nodejs version is recommended. I see now that I use an old version of nodejs.

    0.12.6 <-- this is my old version
    4.4.1 <-- this is the one I will update to

    Release: Gentoo

    Maybe thats the reason 😛 I will try it out in a few days!

    @Diego-Bao Yes, indeed. > v1.0.2 is pretty useless hehe. Trashcan next in my case. Must find the issue 😉 And NO, this has nothing todo with new installation. I having trouble with all my old installations. Upgrade breaks the db! And it seams to be broken even if i downgrade so plz backup before upgrade !

  • GNU/Linux Admin

    Can either of you send a database dump to me? (my username @ nodebb.org)

    Perhaps it will reproduce on my local environment... but without traffic, I doubt it...

  • Swedes

    Yes, I will try to use nodejs 4x first and if that dont work i then i can send files and db dump to you 🙂 Traffic is not the issue in my case, no traffic but it still hangs 😛


  • @julian Sorry I previously said I was running on master but I'm running on v1.x.x branch

  • Swedes

    I have it up and running again at v1.x.x branch

    • Mongo 2.6
    • Nodejs 4.4.1

    Lets see how long it takes before hell breaks out again 😛

    Edit: Still running but even if this does not work its recommended to upgrade to Nodejs 4.4.1 🙂 Seams a lot faster 😛


  • @Jenkler it shouldn't take long lately it just takes me half an hour to break.

  • Swedes

    Wiiii, it seams that the problem is solved for me. I will test more today but no crash since yesterday.

    The issue was with Nodejs 0.12.6, I upgraded to Nodejs 4.4.1 and now it works.

    Thanks for all input ❤ I love NodeBB !


  • I upgraded to Nodejs 4.4.5 and running Nodebb v1.0.3 (master). in less that 30 mins it froze again.


  • I completely uninstall it and did a clean instalation of the code (left the dB alone). Installed version v1.0.2 and so far it's running fine.

    Only thing I noticed is that the notifications number in my user shows 1, when I don't have any notification. I mark all as read, but if I refresh the page I get 1 again. I will keep updating here if I have more issues.

Suggested Topics

| |