Solved Forum down after restart


  • @baris @PitaJ just tried an upgrade, and got this

    Welcome to Ubuntu 20.04.3 LTS (GNU/Linux 5.13.7-051307-lowlatency x86_64)
    
     * Documentation:  https://help.ubuntu.com
     * Management:     https://landscape.canonical.com
     * Support:        https://ubuntu.com/advantage
    Last login: Thu Sep 23 19:49:08 2021 from 77.101.200.87
    [email protected]:~$ cd nodebb
    [email protected]:~/nodebb$ git fetch
    remote: Enumerating objects: 767, done.
    remote: Counting objects: 100% (607/607), done.
    remote: Compressing objects: 100% (91/91), done.
    remote: Total 767 (delta 535), reused 580 (delta 516), pack-reused 160
    Receiving objects: 100% (767/767), 196.89 KiB | 12.30 MiB/s, done.
    Resolving deltas: 100% (576/576), completed with 254 local objects.
    From https://github.com/NodeBB/NodeBB
     * [new branch]          dependabot/npm_and_yarn/install/commander-8.2.0         -> origin/dependabot/npm_and_yarn/install/commander-8.2.0
     + e0fda82b2...7502d5da8 dependabot/npm_and_yarn/install/connect-pg-simple-7.0.0 -> origin/dependabot/npm_and_yarn/install/connect-pg-simple-7.0.0  (forced update)
     * [new branch]          dependabot/npm_and_yarn/install/mongodb-4.1.2           -> origin/dependabot/npm_and_yarn/install/mongodb-4.1.2
     + b0787b593...a8ba14cdc dependabot/npm_and_yarn/install/postcss-clean-1.2.2     -> origin/dependabot/npm_and_yarn/install/postcss-clean-1.2.2  (forced update)
       0a56158bb..04ee5c0df  master                                                  -> origin/master
     + 4a5f10d2a...b638183dd renovate/mongodb-4.x                                    -> origin/renovate/mongodb-4.x  (forced update)
     * [new branch]          topic-linkbacks                                         -> origin/topic-linkbacks
       9410a73aa..1fad6f680  v1.17.x                                                 -> origin/v1.17.x
       854c078b7..9e5223697  v1.18.x                                                 -> origin/v1.18.x
     * [new tag]             v1.17.2-beta.8                                          -> v1.17.2-beta.8
     * [new tag]             v1.18.3                                                 -> v1.18.3
     * [new tag]             v1.17.2-beta.3                                          -> v1.17.2-beta.3
     * [new tag]             v1.17.2-beta.4                                          -> v1.17.2-beta.4
     * [new tag]             v1.17.2-beta.5                                          -> v1.17.2-beta.5
     * [new tag]             v1.17.2-beta.6                                          -> v1.17.2-beta.6
     * [new tag]             v1.17.2-beta.7                                          -> v1.17.2-beta.7
    [email protected]:~/nodebb$ git reset --hard origin/v1.18.x
    HEAD is now at 9e5223697 chore: incrementing version number - v1.18.3
    [email protected]:~/nodebb$ ./nodebb upgrade
    
    Updating NodeBB...
    
    1. Updating package.json file with defaults...  OK
    
    2. Bringing base dependencies up to date...  started
    npm ERR! code ERESOLVE
    npm ERR! ERESOLVE unable to resolve dependency tree
    npm ERR! 
    npm ERR! While resolving: [email protected]
    npm ERR! Found: [email protected]
    npm ERR! node_modules/mongodb
    npm ERR!   [email protected]"3.7.1" from the root project
    npm ERR! 
    npm ERR! Could not resolve dependency:
    npm ERR! peer [email protected]"^4.1.0" from [email protected]
    npm ERR! node_modules/connect-mongo
    npm ERR!   [email protected]"4.6.0" from the root project
    npm ERR! 
    npm ERR! Fix the upstream dependency conflict, or retry
    npm ERR! this command with --force, or --legacy-peer-deps
    npm ERR! to accept an incorrect (and potentially broken) dependency resolution.
    npm ERR! 
    npm ERR! See /home/sudonix/.npm/eresolve-report.txt for a full report.
    
    npm ERR! A complete log of this run can be found in:
    npm ERR!     /home/sudonix/.npm/_logs/2021-09-23T18_51_40_851Z-debug.log
    Error installing dependencies!
    message: Command failed: npm install --production
    stdout: null
    stderr: null
    Error occurred during upgrade: Error: Command failed: npm install --production
        at checkExecSyncError (node:child_process:826:11)
        at Object.execSync (node:child_process:900:15)
        at Object.installAll (/home/sudonix/nodebb/src/cli/package-install.js:73:9)
        at Object.handler (/home/sudonix/nodebb/src/cli/upgrade.js:21:19)
        at runSteps (/home/sudonix/nodebb/src/cli/upgrade.js:54:16)
        at processTicksAndRejections (node:internal/process/task_queues:96:5)
        at async Object.runUpgrade [as upgrade] (/home/sudonix/nodebb/src/cli/upgrade.js:84:3)
    2021-09-23T18:51:40.893Z [4567/78126] - error: uncaughtException: Command failed: npm install --production
    Error: Command failed: npm install --production
        at checkExecSyncError (node:child_process:826:11)
        at Object.execSync (node:child_process:900:15)
        at Object.installAll (/home/sudonix/nodebb/src/cli/package-install.js:73:9)
        at Object.handler (/home/sudonix/nodebb/src/cli/upgrade.js:21:19)
        at runSteps (/home/sudonix/nodebb/src/cli/upgrade.js:54:16)
        at processTicksAndRejections (node:internal/process/task_queues:96:5)
        at async Object.runUpgrade [as upgrade] (/home/sudonix/nodebb/src/cli/upgrade.js:84:3) {"error":{"status":1,"signal":null,"output":[null,null,null],"pid":78133,"stdout":null,"stderr":null},"stack":"Error: Command failed: npm install --production\n    at checkExecSyncError (node:child_process:826:11)\n    at Object.execSync (node:child_process:900:15)\n    at Object.installAll (/home/sudonix/nodebb/src/cli/package-install.js:73:9)\n    at Object.handler (/home/sudonix/nodebb/src/cli/upgrade.js:21:19)\n    at runSteps (/home/sudonix/nodebb/src/cli/upgrade.js:54:16)\n    at processTicksAndRejections (node:internal/process/task_queues:96:5)\n    at async Object.runUpgrade [as upgrade] (/home/sudonix/nodebb/src/cli/upgrade.js:84:3)","exception":true,"date":"Thu Sep 23 2021 19:51:40 GMT+0100 (British Summer Time)","process":{"pid":78126,"uid":1021,"gid":1011,"cwd":"/home/sudonix/nodebb","execPath":"/usr/bin/node","version":"v16.10.0","argv":["/usr/bin/node","/home/sudonix/nodebb/nodebb","upgrade"],"memoryUsage":{"rss":67944448,"heapTotal":31772672,"heapUsed":21097072,"external":1831674,"arrayBuffers":926654}},"os":{"loadavg":[0.27,0.12,0.03],"uptime":85792.33},"tr:ace":[{"column":11,"file":"node:child_process","function":"checkExecSyncError","line":826,"method":null,"native":false},{"column":15,"file":"node:child_process","function":"Object.execSync","line":900,"method":"execSync","native":false},{"column":9,"file":"/home/sudonix/nodebb/src/cli/package-install.js","function":"Object.installAll","line":73,"method":"installAll","native":false},{"column":19,"file":"/home/sudonix/nodebb/src/cli/upgrade.js","function":"Object.handler","line":21,"method":"handler","native":false},{"column":16,"file":"/home/sudonix/nodebb/src/cli/upgrade.js","function":"runSteps","line":54,"method":null,"native":false},{"column":5,"file":"node:internal/process/task_queues","function":"processTicksAndRejections","line":96,"method":null,"native":false},{"column":3,"file":"/home/sudonix/nodebb/src/cli/upgrade.js","function":"async Object.runUpgrade [as upgrade]","line":84,"method":"runUpgrade [as upgrade]","native":false}]}
    [email protected]:~/nodebb$ 
    

    😟

    Any thoughts ?

    Thanks



  • @baris thanks


  • @baris @pitaj after upgrading to 1.18.3 I now get this when attempting to start NodeBB (as daemon)

        at processTicksAndRejections (node:internal/process/task_queues:84:21)
    2021-09-23T19:01:07.217Z [4567/78763] - info: [app] Shutdown (SIGTERM/SIGINT) Initialised.
    2021-09-23T19:01:07.218Z [4567/78763] - error: Error [ERR_SERVER_NOT_RUNNING]: Server is not running.
        at new NodeError (node:internal/errors:371:5)
        at Server.close (node:net:1613:12)
        at Object.onceWrapper (node:events:509:28)
        at Server.emit (node:events:402:35)
        at emitCloseNT (node:net:1666:8)
        at processTicksAndRejections (node:internal/process/task_queues:82:21)
    3 restarts in 10 seconds, most likely an error on startup. Halting.
    

  • Starting with "daemon": false works, but I'd rather use the daemon

  • NodeBB Admin

    What version of node & npm are you using now?


  • @baris Versions below
    Node.js v16.10.0.
    npm --version 6.14.15

  • NodeBB Admin

    Can you try latest node 14.x, we don't official support 16 yet.


  • @baris ok. How can I downgrade ?

  • NodeBB Admin


  • @baris thanks. I removed NodeJS and reinstalled it using 14.x version. NodeBB now looks like it starts using the daemon (according to the logs, although if I try nodebb stop it tells me NodeBB is already stopped), and I get 502 bad gateway - but nothing has changed in terms of overall directory structure and nginx config.

    So it looks like whilst the original odd message has cleared, it's not starting

    My other NodeBB site works fine


  • @phenomlab could you try deleting your log file and starting with ./nodebb slog again?


  • @pitaj doesn't seem to do anything other than this

    Starting NodeBB with logging output
    Hit Ctrl-C to exit
    The NodeBB process will continue to run in the background
    Use "./nodebb stop" to stop the NodeBB server
    tail: cannot open './logs/output.log' for reading: No such file or directory
    tail: './logs/output.log' has appeared;  following new file
    

  • At this point, I'm totally stumped. The only other option I have available is to create a new instance and attach the current database to that to see if this resolves the issue or not.

    Failing that, I'm at a total loss here.


  • @PitaJ @baris I've been looking into this more, and but when I start NodeBB I get this message

    2021-09-26T19:52:22.547Z [4567/349055] - info: Enabling 'trust proxy'
    2021-09-26T19:52:22.550Z [4567/349055] - error: NodeBB address in use, exiting...
    Error: listen EADDRINUSE: address already in use 0.0.0.0:4567
        at Server.setupListenHandle [as _listen2] (net.js:1320:16)
        at listenInCluster (net.js:1368:12)
        at doListen (net.js:1505:7)
        at processTicksAndRejections (internal/process/task_queues.js:83:21)
    2021-09-26T19:52:22.552Z [4567/349055] - error: uncaughtException: listen EADDRINUSE: address already in use 0.0.0.0:4567
    Error: listen EADDRINUSE: address already in use 0.0.0.0:4567
    3 restarts in 10 seconds, most likely an error on startup. Halting.
    

    The problem is, the port isn't actively being used anywhere. I have another NodeBB instance running on 4568, but that's it.

    Any thoughts ?

    Thanks


  • @phenomlab I don't know what to tell you. It's not lying about the address being in use.

    Does it work if you run node app? If not, it's definitely a rogue process left running on that port.


  • @pitaj thanks. Found the rogue process, and now I'm stuck with this.

    [email protected]:~/nodebb$ ./nodebb start && ./nodebb log
    
    Starting NodeBB
      "./nodebb stop" to stop the NodeBB server
      "./nodebb log" to view server output
      "./nodebb help" for more commands
    
    
    Hit Ctrl-C to exit
    
    
        at processTicksAndRejections (internal/process/task_queues.js:83:21)
    2021-09-26T20:43:20.339Z [4567/2900] - info: [app] Shutdown (SIGTERM/SIGINT) Initialised.
    2021-09-26T20:43:20.340Z [4567/2900] - error: Error [ERR_SERVER_NOT_RUNNING]: Server is not running.
        at Server.close (net.js:1608:12)
        at Object.onceWrapper (events.js:519:28)
        at Server.emit (events.js:412:35)
        at emitCloseNT (net.js:1661:8)
        at processTicksAndRejections (internal/process/task_queues.js:81:21)
    2021-09-26T20:43:20.346Z [4567/2893] - info: [reputation-rules] settings loaded
    3 restarts in 10 seconds, most likely an error on startup. Halting.
    

    So it looks like the process spawns and then immediately crashes. Trouble is I don't know where. It even does this with all plugins disabled.


  • @pitaj said in Forum down after restart:

    Does it work if you run node app?

    Yes

    [email protected]:~/nodebb$ node app
    2021-09-26T21:37:41.594Z [4567/5912] - info: NodeBB v1.18.3 Copyright (C) 2013-2021 NodeBB Inc.
    2021-09-26T21:37:41.596Z [4567/5912] - info: This program comes with ABSOLUTELY NO WARRANTY.
    2021-09-26T21:37:41.596Z [4567/5912] - info: This is free software, and you are welcome to redistribute it under certain conditions.
    2021-09-26T21:37:41.596Z [4567/5912] - info: 
    2021-09-26T21:37:41.597Z [4567/5912] - info: Initializing NodeBB v1.18.3 https://sudonix.com
    2021-09-26T21:37:43.378Z [4567/5912] - info: [socket.io] Restricting access to origin: https://sudonix.com:*
    
    2021-09-26T21:37:43.808Z [4567/5912] - warn: [plugins/load] The following plugins may not be compatible with your version of NodeBB. This may cause unintended behaviour or crashing. In the event of an unresponsive NodeBB caused by this plugin, run `./nodebb reset -p PLUGINNAME` to disable it.
      * nodebb-plugin-emoji
    
    2021-09-26T21:37:43.931Z [4567/5912] - info: [plugins/spam-be-gone] Settings loaded
    2021-09-26T21:37:43.943Z [4567/5912] - info: [reputation-rules] settings loaded
    2021-09-26T21:37:43.971Z [4567/5912] - info: [api] Adding 1 route(s) to `api/v3/plugins`
    2021-09-26T21:37:43.999Z [4567/5912] - info: [router] Routes added
    2021-09-26T21:37:44.007Z [4567/5912] - info: NodeBB Ready
    2021-09-26T21:37:44.009Z [4567/5912] - info: Enabling 'trust proxy'
    2021-09-26T21:37:44.013Z [4567/5912] - info: NodeBB is now listening on: 0.0.0.0:4567
    

  • Does it still work with daemon:false or does that no longer help?


  • @pitaj that works fine

    The only issue of course is that when performing a restart from the ACP I have to kill the process and restart it manually


Suggested Topics

| | | |