nodebb slows down

Technical Support
  • Actually we have two problems:

    a.The problem I mentioned at first which is about NodeBB performance . While NodeBB is running, it gets slower and slower till we have to restart it to make it fresh . In fact we have created a cron-job on our server for that . It restarts NodeBB every 6 hours and according to our experience when the time is close to a NodeBB restart, it has become really slow and it needs a restart vary badly.

    b.The second problem is about some NodeBB bugs that has been happening since we changed our Nginx configuration and ran it on three ports instead of just one port. After doing this we are facing a bunch of came-out-of-nowhere problems. Like sometimes when we open our forum's first page we can't see any categories (it is blank), or when we click on unseen posts it shows a route error instead. In the other words our forum doesn't work normally now and I'm sure you can imagine how frustrating it is. The tricky point here is that we also have Varnish. So there is NodeBB, Nginx and a Varnish in between which has been configured in the same way the NodeBB tutorials says. It basically passes every socket.io requests to NodeBB directly and every requests for a static file to the Nginx. But we have had Varnish for a while and we hadn't experienced any problems before we changes Nginx.

    Lastly, we appreciate any helps from you guys.
    Thanks in advance.

  • I just changed Varnish configuration.I added these three blocks:
    at the begining:

    sub vcl_init {                                                                                                                                                    
        new nodebb = directors.round_robin();                                                                                                                         
        nodebb.add_backend(nodebb01);                                                                                                                                 
        nodebb.add_backend(nodebb02);                                                                                                                                 
        nodebb.add_backend(nodebb03);                                                                                                                                 
    }  
    

    and in the middle:

    # Pipe websocket connections directly to Node.js                                                                                                            
          if (req.http.Upgrade ~ "(?i)websocket") {                                                                                                                   
             set req.backend_hint = nodebb.backend();                                                                                                                 
             return (pipe);                                                                                                                                           
          } 
    
    set req.backend_hint = forum;                                                                                                                     
    			if (req.url ~ "^/socket.io/") {                                                                                                               
    			  set req.backend_hint = nodebb.backend();                                                                                                    
    			  return (pipe); # return pass seems not working for websockets                                                                               
    			} 
    
  • What's the swap config like on your machine?

  • @julian I don't know, please tell me how can I know that?
    We are using debian 8 on SSD hard drive with 16GB memory and 24 core(intel xenon) cpu that serve multiple php websites and only one nodejs app (=NodeBB) .

    I have checked the server usage we are using 8G memory and 50% cpu usage at most!(in worse case )

  • @sanatisharif That is quite peculiar. I have not seen these symptoms before. NodeBB has always been fairly snappy and does not slow down with time...

  • @julian maybe nodebb acts like this because of varnish configs.

    I have changed it to hash directors instead of round_robin with 'client.identifier' as ID some of our problems gone but users complicate that some times our comunity is very slow!

    sub vcl_init {                                                                                                                                                    
        new nodebb = directors.hash();                                                                                                                         
        nodebb.add_backend(nodebb01);                                                                                                                                 
        nodebb.add_backend(nodebb02);                                                                                                                                 
        nodebb.add_backend(nodebb03);                                                                                                                                 
    }
    
    set req.backend_hint = nodebb.backend(client.identity);
    
  • Also when I checked varnish log dozens of request will be passed(MISS) so we can have better config for varnish.

  • @sanatisharif said in nodebb slows down:

    We are using debian 8 on SSD hard drive with 16GB memory and 24 core(intel xenon) cpu that serve multiple php websites and only one nodejs app (=NodeBB) .

    How many NodeBB processes are you running? Are they utilising a lot of CPU?

  • @julian said in nodebb slows down:

    @sanatisharif said in nodebb slows down:

    We are using debian 8 on SSD hard drive with 16GB memory and 24 core(intel xenon) cpu that serve multiple php websites and only one nodejs app (=NodeBB) .

    How many NodeBB processes are you running? Are they utilising a lot of CPU?

    I don't know, please tell me how can I know that?

  • You can check top for that.

  • @sanatisharif said in nodebb slows down:

    I don't know, please tell me how can I know that?

    $> top
    

    this will give you a bunch of statistics about the system, including that's running and how much CPU it is taking.

    to find out how many processes of nodejs are running i would recommend using

    #> ps aux |grep node
    

    this will spit our information about all processes that have "node" in their command line. you'll probably get a few false positives but you should be able to identify how many of those are nodebb easily enough.


Suggested Topics


  • 0 Votes
    7 Posts
    5k Views

    @pitaj

    The plugin was indeed in the dependencies section even though it was not installed on my nodeBB instance, weird.

    Removing it has fixed the error. Thank you so much for your help 🙂

    Here's how the package.json file looked like:

    { "name": "nodebb", "license": "GPL-3.0", "description": "NodeBB Forum", "version": "1.16.1", "homepage": "http://www.nodebb.org", "repository": { "type": "git", "url": "https://github.com/NodeBB/NodeBB/" }, "main": "app.js", "scripts": { "start": "node loader.js", "lint": "npx eslint --cache ./nodebb .", "test": "npx nyc --reporter=html --reporter=text-summary npx mocha", "coverage": "nyc report --reporter=text-lcov > ./coverage/lcov.info", "coveralls": "nyc report --reporter=text-lcov | coveralls && rm -r coverage" }, "husky": { "hooks": { "pre-commit": "npx lint-staged", "commit-msg": "npx commitlint -E HUSKY_GIT_PARAMS" } }, "lint-staged": { "*.js": [ "eslint --fix", "git add" ] }, "dependencies": { "@adactive/bootstrap-tagsinput": "^0.8.2", "@nodebb/bootswatch": "3.4.2", "ace-builds": "^1.4.9", "archiver": "^5.0.0", "async": "^3.2.0", "autoprefixer": "10.2.0", "bcryptjs": "2.4.3", "benchpressjs": "2.4.0", "body-parser": "^1.19.0", "bootbox": "4.4.0", "bootstrap": "^3.4.1", "chart.js": "^2.9.3", "cli-graph": "^3.2.2", "clipboard": "^2.0.6", "colors": "^1.4.0", "commander": "^6.0.0", "compare-versions": "3.6.0", "compression": "^1.7.4", "connect-ensure-login": "^0.1.1", "connect-flash": "^0.1.1", "connect-mongo": "3.2.0", "connect-multiparty": "^2.2.0", "connect-pg-simple": "^6.1.0", "connect-redis": "5.0.0", "cookie-parser": "^1.4.5", "cron": "^1.8.2", "cropperjs": "^1.5.6", "csurf": "^1.11.0", "daemon": "^1.1.0", "diff": "^5.0.0", "express": "^4.17.1", "express-session": "^1.17.0", "express-useragent": "^1.0.13", "graceful-fs": "^4.2.3", "helmet": "^4.0.0", "html-to-text": "6.0.0", "ipaddr.js": "^2.0.0", "jquery": "3.5.1", "jquery-deserialize": "2.0.0-rc1", "jquery-form": "4.3.0", "jquery-serializeobject": "1.0.0", "jquery-ui": "1.12.1", "jsesc": "3.0.2", "json2csv": "5.0.5", "jsonwebtoken": "^8.5.1", "less": "^3.11.1", "lodash": "^4.17.15", "logrotate-stream": "^0.2.6", "lru-cache": "6.0.0", "material-design-lite": "^1.3.0", "mime": "^2.4.4", "mkdirp": "^1.0.4", "mongodb": "3.6.3", "morgan": "^1.10.0", "mousetrap": "^1.6.5", "nconf": "^0.11.0", "nodebb-plugin-composer-default": "6.5.5", "nodebb-plugin-dbsearch": "4.1.2", "nodebb-plugin-emoji": "^3.3.0", "nodebb-plugin-emoji-android": "2.0.0", "nodebb-plugin-gamification": "^1.0.1", "nodebb-plugin-markdown": "8.12.4", "nodebb-plugin-mentions": "2.13.6", "nodebb-plugin-question-and-answer": "^0.7.7", "nodebb-plugin-require": "0.0.6", "nodebb-plugin-seo": "^1.0.0", "nodebb-plugin-seo-slug-friendly": "0.0.8", "nodebb-plugin-seo-thread": "^0.1.10", "nodebb-plugin-soundpack-default": "1.0.0", "nodebb-plugin-spam-be-gone": "0.7.7", "nodebb-rewards-essentials": "0.1.4", "nodebb-theme-lavender": "5.0.17", "nodebb-theme-persona": "10.3.17", "nodebb-theme-slick": "1.3.7", "nodebb-theme-vanilla": "11.3.10", "nodebb-widget-essentials": "5.0.2", "nodemailer": "^6.4.6", "nprogress": "0.2.0", "passport": "^0.4.1", "passport-http-bearer": "^1.0.1", "passport-local": "1.0.0", "pg": "^8.0.2", "pg-cursor": "^2.1.9", "postcss": "8.1.10", "postcss-clean": "1.1.0", "promise-polyfill": "^8.1.3", "prompt": "^1.0.0", "redis": "3.0.2", "request": "2.88.2", "request-promise-native": "^1.0.8", "requirejs": "2.3.6", "rimraf": "3.0.2", "rss": "^1.2.2", "sanitize-html": "^2.0.0", "semver": "^7.2.1", "serve-favicon": "^2.5.0", "sharp": "0.27.0", "sitemap": "^6.1.0", "slideout": "1.0.1", "socket.io": "3.0.5", "socket.io-adapter-cluster": "^1.0.1", "socket.io-client": "3.0.5", "socket.io-redis": "6.0.1", "sortablejs": "1.10.2", "spdx-license-list": "^6.1.0", "spider-detector": "2.0.0", "textcomplete": "^0.17.1", "textcomplete.contenteditable": "^0.1.1", "timeago": "^1.6.7", "tinycon": "0.6.8", "toobusy-js": "^0.5.1", "uglify-es": "^3.3.9", "validator": "13.5.2", "visibilityjs": "2.0.2", "winston": "3.3.3", "xml": "^1.0.1", "xregexp": "^4.3.0", "zxcvbn": "^4.4.2" }, "devDependencies": { "@apidevtools/swagger-parser": "10.0.2", "@commitlint/cli": "11.0.0", "@commitlint/config-angular": "11.0.0", "coveralls": "3.1.0", "eslint": "7.17.0", "eslint-config-airbnb-base": "14.2.1", "eslint-plugin-import": "2.22.1", "grunt": "1.3.0", "grunt-contrib-watch": "1.1.0", "husky": "4.3.6", "jsdom": "16.4.0", "lint-staged": "10.5.3", "mocha": "8.2.1", "mocha-lcov-reporter": "1.3.0", "nyc": "15.1.0", "smtp-server": "3.8.0" }, "bugs": { "url": "https://github.com/NodeBB/NodeBB/issues" }, "engines": { "node": ">=10" }, "maintainers": [ { "name": "Andrew Rodrigues", "email": "[email protected]", "url": "https://github.com/psychobunny" }, { "name": "Julian Lam", "email": "[email protected]", "url": "https://github.com/julianlam" }, { "name": "Barış Soner Uşaklı", "email": "[email protected]", "url": "https://github.com/barisusakli" } ], "nyc": { "exclude": [ "src/upgrades/*", "test/*" ] } }
  • 0 Votes
    4 Posts
    565 Views

    Problem solved. For this topic to be any usefull at all:

    Run into first problem trying to point and click install 2 nodebb instances( test and the actual) on the same server and obviously, both couldn't listen on the same 4567 port.

    Second, did the second install from the old docs on readthedocs instead of the official ones from nodebb.

    Third, forgot to also add php-fpm directives to my Nginx config. And start Mongodb.

  • Debug Nodebb load

    Technical Support
    0 Votes
    15 Posts
    2k Views

    @damian-gądziak Maybe you can try to debug by adding this in that function console.log(tids.length, uid, new Error('test').stack);

    Topics.filterWatchedTids = function (tids, uid, callback) { console.log(tids.length, uid, new Error('test').stack); async.waterfall([ function (next) { db.sortedSetScores('uid:' + uid + ':followed_tids', tids, next); }, function (scores, next) { tids = tids.filter(function (tid, index) { return tid && !!scores[index]; }); next(null, tids); }, ], callback); };

    I am guessing the function is called by a huge amount of tids, so the next step would be to find out why and from where.

  • 0 Votes
    23 Posts
    5k Views

    The new systemctl block in the documentation works wonderfully.

    The ONE time that NodeBB went down on my forums, I was on vacation in Mexico. Logging into the server from my phone was atrocious. Now systemctl can do all the work for me 😄

  • 0 Votes
    3 Posts
    1k Views

    @julian Ah i see, any idea how i can fix that? I've followed the guide at https://docs.nodebb.org/installing/os/ubuntu/ and created the users exactly like the guide said.