Frontend freeze on version later than v1.0.2

  • Community Rep

    I've started seeing this behavior as well on my dev environment, maybe it's related?

    Running 1.x.x branch latest as of May 20ish (1.0.3 @ a196c834d9aab4e1c27965dcca623ce12e61b4ae), mongo 3.0.7

    I'll be browsing around, and can see about 10 connections in total opened up in mongo, and then after some variable number of minutes, the application just becomes unresponsive; no error logs from node bb or anything. However, I see logs from mongo like this (note the timestamps):

    2016-05-29T21:38:52.811-0700 I ACCESS   [conn11] Successfully authenticated as principal nodebb on nodebb
    
    // then, after about 19 minutes:
    2016-05-29T21:57:19.778-0700 I NETWORK  [conn2] end connection 127.0.0.1:52223 (9 connections now open)
    2016-05-29T21:57:19.778-0700 I NETWORK  [conn7] end connection 127.0.0.1:52234 (8 connections now open)
    2016-05-29T21:57:19.779-0700 I NETWORK  [conn9] end connection 127.0.0.1:52236 (7 connections now open)
    2016-05-29T21:57:19.779-0700 I NETWORK  [conn4] end connection 127.0.0.1:52231 (6 connections now open)
    2016-05-29T21:57:19.779-0700 I NETWORK  [conn8] end connection 127.0.0.1:52235 (5 connections now open)
    2016-05-29T21:57:19.782-0700 I NETWORK  [conn6] end connection 127.0.0.1:52233 (4 connections now open)
    2016-05-29T21:57:23.474-0700 I NETWORK  [conn11] end connection 127.0.0.1:52251 (3 connections now open)
    2016-05-29T21:57:23.474-0700 I NETWORK  [conn3] end connection 127.0.0.1:52224 (3 connections now open)
    2016-05-29T21:57:23.474-0700 I NETWORK  [conn5] end connection 127.0.0.1:52232 (1 connection now open)
    2016-05-29T21:57:28.231-0700 I NETWORK  [conn10] end connection 127.0.0.1:52237 (0 connections now open)
    

    @Jenkler have you found mongo logs like this at all?

    I can't think of any plugins that I have installed that would cause this, the only thing I've been changing is the theme and the composer plugin, and some fresh npm installs... I haven't been switching the NBB branch at all.

  • Community Rep

    It seems pretty random, but here's another sample:

    2016-05-29T22:58:13.486-0700 I NETWORK  [initandlisten] connection accepted from 127.0.0.1:53521 #31 (1 connection now open)
    2016-05-29T22:58:13.561-0700 I ACCESS   [conn31] Successfully authenticated as principal nodebb on nodebb
    2016-05-29T23:02:00.357-0700 I NETWORK  [conn31] end connection 127.0.0.1:53521 (0 connections now open)
    2016-05-29T23:02:01.362-0700 I NETWORK  [initandlisten] connection accepted from 127.0.0.1:53603 #32 (1 connection now open)
    2016-05-29T23:02:01.440-0700 I ACCESS   [conn32] Successfully authenticated as principal nodebb on nodebb
    2016-05-29T23:04:03.492-0700 I NETWORK  [initandlisten] connection accepted from 127.0.0.1:53632 #33 (2 connections now open)
    2016-05-29T23:04:03.518-0700 I NETWORK  [initandlisten] connection accepted from 127.0.0.1:53634 #34 (3 connections now open)
    2016-05-29T23:04:23.842-0700 I NETWORK  [initandlisten] connection accepted from 127.0.0.1:53638 #35 (4 connections now open)
    2016-05-29T23:04:23.846-0700 I NETWORK  [initandlisten] connection accepted from 127.0.0.1:53639 #36 (5 connections now open)
    2016-05-29T23:04:23.846-0700 I NETWORK  [initandlisten] connection accepted from 127.0.0.1:53640 #37 (6 connections now open)
    2016-05-29T23:04:23.846-0700 I NETWORK  [initandlisten] connection accepted from 127.0.0.1:53641 #38 (7 connections now open)
    2016-05-29T23:04:23.848-0700 I NETWORK  [initandlisten] connection accepted from 127.0.0.1:53642 #39 (8 connections now open)
    2016-05-29T23:04:23.849-0700 I NETWORK  [initandlisten] connection accepted from 127.0.0.1:53643 #40 (9 connections now open)
    2016-05-29T23:04:23.850-0700 I NETWORK  [initandlisten] connection accepted from 127.0.0.1:53644 #41 (10 connections now open)
    2016-05-29T23:06:03.511-0700 I NETWORK  [conn33] end connection 127.0.0.1:53632 (9 connections now open)
    2016-05-29T23:06:03.519-0700 I NETWORK  [conn34] end connection 127.0.0.1:53634 (8 connections now open)
    2016-05-29T23:06:23.844-0700 I NETWORK  [conn35] end connection 127.0.0.1:53638 (7 connections now open)
    2016-05-29T23:06:23.844-0700 I NETWORK  [conn36] end connection 127.0.0.1:53639 (7 connections now open)
    2016-05-29T23:06:23.844-0700 I NETWORK  [conn37] end connection 127.0.0.1:53640 (5 connections now open)
    2016-05-29T23:06:23.844-0700 I NETWORK  [conn38] end connection 127.0.0.1:53641 (4 connections now open)
    2016-05-29T23:06:23.844-0700 I NETWORK  [conn39] end connection 127.0.0.1:53642 (3 connections now open)
    2016-05-29T23:06:23.847-0700 I NETWORK  [conn40] end connection 127.0.0.1:53643 (2 connections now open)
    2016-05-29T23:06:23.847-0700 I NETWORK  [conn41] end connection 127.0.0.1:53644 (1 connection now open)
    2016-05-29T23:07:06.570-0700 I NETWORK  [conn32] end connection 127.0.0.1:53603 (0 connections now open)
    
  • Swedes

    @julian No, that one i can try. I have now reverted all my dbs but i can try to setup the same stuff aging with v1.0.3 and my db.

    1. I will try the ./nodebb reset -p command
    2. I will check my mongodb logs!

    It would be nice to solve this 🙂

    Edit1: Dev setup with my db and v1.x.x is now done. Will wait for first crash. Also looking in mongodb.log 😉
    Edit2: Don know why but i get 4 auth instead of 1 when the site auths to the db. Normal ? My v1.0.2 sites only get 1 every time 🙂
    2016-05-30T11:53:10.080+0200 [conn75] authenticate db: dev_nodebb_se { authenticate: 1, user: "dev_nodebb_se", nonce: "xxx", key: "xxx" }
    2016-05-30T11:53:10.085+0200 [conn76] authenticate db: dev_nodebb_se { authenticate: 1, user: "dev_nodebb_se", nonce: "xxx", key: "xxx" }
    2016-05-30T11:53:10.087+0200 [conn77] authenticate db: dev_nodebb_se { authenticate: 1, user: "dev_nodebb_se", nonce: "xxx", key: "xxx" }
    2016-05-30T11:55:13.907+0200 [conn78] authenticate db: dev_nodebb_se { authenticate: 1, user: "dev_nodebb_se", nonce: "xxx", key: "xxx" }
    Edit3: First crash, nothing in mongodblogs and this in nodebb dev
    30/5 11:51 [68] - warn: [plugins] Expected method for hook 'filter:admin.header.build' in plugin 'nodebb-plugin-youtube-lite' not found, skipping.
    30/5 11:52 [68] - verbose: [translator] No resource file found for sv/markdown, using provided fallback language file
    30/5 11:52 [68] - warn: Route requested but not found: /vendor/jquery/timeago/locales/jquery.timeago.sv-short.js?_=1464602028233
    info: Imgbed: cache miss
    info: Imgbed: cache miss
    info: Imgbed: cache miss
    info: Imgbed: cache miss
    info: Imgbed: cache miss
    info: Imgbed: cache miss
    ...
    Edit4 Running ./nodebb reset -p and start nodebb dev again
    Edit5 Crash again. No errors no nothing Running ./nodebb reset -a and start nodebb dev again
    Edit6 To many db connections if you ask me 😉 Something seams wrong here!
    2016-05-30T12:20:34.571+0200 [conn115] authenticate db: dev_nodebb_se { authenticate: 1, user: "dev_nodebb_se", nonce: "xxx", key: "xxx" }
    2016-05-30T12:20:34.575+0200 [conn116] authenticate db: dev_nodebb_se { authenticate: 1, user: "dev_nodebb_se", nonce: "xxx", key: "xxx" }
    2016-05-30T12:20:34.579+0200 [conn117] authenticate db: dev_nodebb_se { authenticate: 1, user: "dev_nodebb_se", nonce: "xxx", key: "xxx" }
    2016-05-30T12:20:34.589+0200 [conn118] authenticate db: dev_nodebb_se { authenticate: 1, user: "dev_nodebb_se", nonce: "xxx", key: "xxx" }
    2016-05-30T12:20:34.592+0200 [conn119] authenticate db: dev_nodebb_se { authenticate: 1, user: "dev_nodebb_se", nonce: "xxx", key: "xxx" }
    2016-05-30T12:20:34.634+0200 [conn120] authenticate db: dev_nodebb_se { authenticate: 1, user: "dev_nodebb_se", nonce: "xxx", key: "xxx" }
    2016-05-30T12:20:38.577+0200 [conn121] authenticate db: dev_nodebb_se { authenticate: 1, user: "dev_nodebb_se", nonce: "xxx", key: "xxx" }
    Edit: Crash again and nothing more to test. Downgrading to v1.0.2 and see if it works again 🙂
    Edit: If I downgrade to v1.0.2 and restore the plugins nodebb crashes. This mean that the upgrade over v1.0.2 fucks your DB in some way. Now i have a broken v1.0.2. Not that my original v1.0.2 works fine on the same system. This is a major BUG!

    Something is wrong, but i cant find out what! More stuff i can try?


  • I'm having the same problem as well.

    I have checked all the things people has mentioned in this topic and several ones.

    1. Disabling plug-in
    2. Check memory (I have 1GB and 2GB of swap memory)
    3. no errors in nodebb logs and mongodb logs.

    Anything I tried is not working.

    If the problem is with version 1.0.3, is there a way to downgrade to 1.0.2 without loosing data?

    Thanks.

  • GNU/Linux Admin

    Are any of you up for running git-bisect to determine which commit started causing these issues?

    Because it only seems to occur on your instances, we cannot reproduce it and find a fix 😦

    We're running master branch here, with no issues.

    That said, do these errors occur on master?

  • Swedes

    Yes, same issue at master!


  • @julian I have installed nodebb less than a couple of weeks ago. I got master branch. Perhaps this only happens with new installations?

    At this point the forums is useless 😞 sometimes it freezes after half an hour, sometimes it runs no problem for a few hours or a whole night.

    If I somehow can help you diagnose the problem, tell me what I can do.

  • Swedes

    I am updating my system right now and wondering what nodejs version is recommended. I see now that I use an old version of nodejs.

    0.12.6 <-- this is my old version
    4.4.1 <-- this is the one I will update to

    Release: Gentoo

    Maybe thats the reason 😛 I will try it out in a few days!

    @Diego-Bao Yes, indeed. > v1.0.2 is pretty useless hehe. Trashcan next in my case. Must find the issue 😉 And NO, this has nothing todo with new installation. I having trouble with all my old installations. Upgrade breaks the db! And it seams to be broken even if i downgrade so plz backup before upgrade !

  • GNU/Linux Admin

    Can either of you send a database dump to me? (my username @ nodebb.org)

    Perhaps it will reproduce on my local environment... but without traffic, I doubt it...

  • Swedes

    Yes, I will try to use nodejs 4x first and if that dont work i then i can send files and db dump to you 🙂 Traffic is not the issue in my case, no traffic but it still hangs 😛


  • @julian Sorry I previously said I was running on master but I'm running on v1.x.x branch

  • Swedes

    I have it up and running again at v1.x.x branch

    • Mongo 2.6
    • Nodejs 4.4.1

    Lets see how long it takes before hell breaks out again 😛

    Edit: Still running but even if this does not work its recommended to upgrade to Nodejs 4.4.1 🙂 Seams a lot faster 😛


  • @Jenkler it shouldn't take long lately it just takes me half an hour to break.

  • Swedes

    Wiiii, it seams that the problem is solved for me. I will test more today but no crash since yesterday.

    The issue was with Nodejs 0.12.6, I upgraded to Nodejs 4.4.1 and now it works.

    Thanks for all input ❤ I love NodeBB !


  • I upgraded to Nodejs 4.4.5 and running Nodebb v1.0.3 (master). in less that 30 mins it froze again.


  • I completely uninstall it and did a clean instalation of the code (left the dB alone). Installed version v1.0.2 and so far it's running fine.

    Only thing I noticed is that the notifications number in my user shows 1, when I don't have any notification. I mark all as read, but if I refresh the page I get 1 again. I will keep updating here if I have more issues.

  • Swedes

    Try 4.4.1 it works for me with v1.x.x


  • The problem with the user notification might be the issue I'm having. If I click to see the notification or unread messages the forum crashes and restarts on its own. But this only happens with my user.
    Well @julian has my DB, so he probably can figure it out what it's happeneing.


  • Hi All,

    Same issue here. Running on Ubuntu 16.04, node.js 4.4.5 and mongodb 2.6. Clean install from scratch (new forum). Everything works nice for about 30min then boom, Nodebb just crashes.... Nothing in Nodebb logs and nothing in MongodDB logs as well...

    I'm going crazy, I've checked almost everything... It's seems like a random behaviour and it crashes with or without activities...

    I've deactivated almost all the plugins... Didn't change anything 😞

    Now I upgraded MongoDB to 3.0 and and changed the 2 kernel params

    ~$ cat /sys/kernel/mm/transparent_hugepage/enabled
    [always] madvise never
    ~$ cat /sys/kernel/mm/transparent_hugepage/defrag
    [always] madvise never

    It's still up for the moment (1h30). I'll keep you posted.


  • Well, it crashes again after 1h45 min 😞 Don't know what to do... Any advice?

Suggested Topics

| |