Facing 504 gateway timeout error while accessing categories and tags in discussion forum
-
Hi team,
We have deployed Nodebb v1.18.6 as microservice in Kubernetes cluster.
After enabling discussions, we are facing below errors-
list itemRequest URL:https://host/discussion/category/13
Request Method:GET
Status Code:504 Gateway Time-out -
list itemRequest URL:* https://host/discussion/forum/tags
-
Request Method:* POST
-
Status Code:* 504 Gateway Time-out
Also, for every request the memory and CPU utilisation is spiking beyond the limits
Please find the logs
2023-07-04T07:41:22.706811749Z [cluster] Child Process (289) has exited (code: null, signal: SIGKILL) 2023-07-04T07:41:22.706961101Z [cluster] Spinning up another process... 2023-07-04T07:41:22.923812885Z 2023-07-04T07:41:22.923Z [4567/300] - ESC[32minfoESC[39m: Initializing NodeBB v1.18.6 http://0.0.0.0:4567/discussions/ 2023-07-04T07:41:24.092229366Z 2023-07-04T07:41:24.092Z [4567/300] - ESC[32minfoESC[39m: [socket.io] Restricting access to origin: http://0.0.0.0:* 2023-07-04T07:41:24.201467001Z 2023-07-04T07:41:24.201Z [4567/300] - ESC[33mwarnESC[39m: [plugins/nodebb-plugin-azurestorage] The plugin.json field "library" is deprecated. Please use the package.json field "main" instead. 2023-07-04T07:41:24.255252846Z 2023-07-04T07:41:24.255Z [4567/300] - ESC[33mwarnESC[39m: [plugins] Unable to load library for: nodebb-plugin-azurestorage 2023-07-04T07:41:24.255492688Z 2023-07-04T07:41:24.255Z [4567/300] - ESC[31merrorESC[39m: Error: Cannot find module './meta' 2023-07-04T07:41:24.255499731Z Require stack: 2023-07-04T07:41:24.255504210Z - /usr/src/app/src/plugins/index.js 2023-07-04T07:41:24.255507646Z - /usr/src/app/src/meta/configs.js 2023-07-04T07:41:24.255511724Z - /usr/src/app/src/meta/index.js 2023-07-04T07:41:24.255515040Z - /usr/src/app/src/start.js 2023-07-04T07:41:24.255518436Z - /usr/src/app/require-main.js 2023-07-04T07:41:24.255521512Z - /usr/src/app/app.js 2023-07-04T07:41:24.255523957Z at Function.Module._resolveFilename (node:internal/modules/cjs/loader:933:15) 2023-07-04T07:41:24.255525700Z at Function.Module._load (node:internal/modules/cjs/loader:778:27) 2023-07-04T07:41:24.255527754Z at Module.require (node:internal/modules/cjs/loader:1005:19) 2023-07-04T07:41:24.255529617Z at Object.<anonymous> (/usr/src/app/node_modules/nodebb-plugin-azurestorage/library.js:14:23) 2023-07-04T07:41:24.255531240Z at Module._compile (node:internal/modules/cjs/loader:1103:14) 2023-07-04T07:41:24.255532874Z at Object.Module._extensions..js (node:internal/modules/cjs/loader:1155:10) 2023-07-04T07:41:24.255534477Z at Module.load (node:internal/modules/cjs/loader:981:32) 2023-07-04T07:41:24.255536020Z at Function.Module._load (node:internal/modules/cjs/loader:822:12) 2023-07-04T07:41:24.255537643Z at Module.require (node:internal/modules/cjs/loader:1005:19) 2023-07-04T07:41:24.255539235Z at require (node:internal/modules/cjs/helpers:102:18) 2023-07-04T07:41:24.339714925Z nodebb plugin sunbird oidc working 2023-07-04T07:41:24.346214244Z 2023-07-04T07:41:24.346Z [4567/300] - ESC[33mwarnESC[39m: [plugins] File not found: /usr/src/app/node_modules/nodebb-plugin-create-forum/public/js/admin.js (Ignoring) 2023-07-04T07:41:24.658189281Z 2023-07-04T07:41:24.658Z [4567/300] - ESC[33mwarnESC[39m: [plugins/nodebb-plugin-telemetry-plugin] The plugin.json field "library" is deprecated. Please use the package.json field "main" instead. 2023-07-04T07:41:24.659193582Z 2023-07-04T07:41:24.659Z [4567/300] - ESC[33mwarnESC[39m: [plugins] Unable to load library for: nodebb-plugin-telemetry-plugin 2023-07-04T07:41:24.659416432Z 2023-07-04T07:41:24.659Z [4567/300] - ESC[31merrorESC[39m: Error: Cannot find module './user' 2023-07-04T07:41:24.659423135Z Require stack: 2023-07-04T07:41:24.659429216Z - /usr/src/app/src/plugins/index.js 2023-07-04T07:41:24.659433163Z - /usr/src/app/src/meta/configs.js 2023-07-04T07:41:24.659436169Z - /usr/src/app/src/meta/index.js 2023-07-04T07:41:24.659437772Z - /usr/src/app/src/start.js 2023-07-04T07:41:24.659439315Z - /usr/src/app/require-main.js 2023-07-04T07:41:24.659441229Z - /usr/src/app/app.js 2023-07-04T07:41:24.659442842Z at Function.Module._resolveFilename (node:internal/modules/cjs/loader:933:15) 2023-07-04T07:41:24.659444845Z at Function.Module._load (node:internal/modules/cjs/loader:778:27) 2023-07-04T07:41:24.659446859Z at Module.require (node:internal/modules/cjs/loader:1005:19) 2023-07-04T07:41:24.659448823Z at Object.<anonymous> (/usr/src/app/node_modules/nodebb-plugin-telemetry-plugin/library.js:1:26) 2023-07-04T07:41:24.659450376Z at Module._compile (node:internal/modules/cjs/loader:1103:14) 2023-07-04T07:41:24.659451919Z at Object.Module._extensions..js (node:internal/modules/cjs/loader:1155:10) 2023-07-04T07:41:24.659453452Z at Module.load (node:internal/modules/cjs/loader:981:32) 2023-07-04T07:41:24.659454985Z at Function.Module._load (node:internal/modules/cjs/loader:822:12) 2023-07-04T07:41:24.659456527Z at Module.require (node:internal/modules/cjs/loader:1005:19) 2023-07-04T07:41:24.659467959Z at require (node:internal/modules/cjs/helpers:102:18) 2023-07-04T07:41:24.659469832Z at Object.Plugins.requireLibrary (/usr/src/app/src/plugins/index.js:62:39) 2023-07-04T07:41:24.659471426Z at registerHooks (/usr/src/app/src/plugins/load.js:160:13) 2023-07-04T07:41:24.659473009Z at Plugins.loadPlugin (/usr/src/app/src/plugins/load.js:123:4) 2023-07-04T07:41:24.659474601Z at async Plugins.reload (/usr/src/app/src/plugins/index.js:112:3) 2023-07-04T07:41:24.659477266Z at async Plugins.init (/usr/src/app/src/plugins/index.js:85:2) 2023-07-04T07:41:24.659480753Z at async initializeNodeBB (/usr/src/app/src/webserver.js:99:2) 2023-07-04T07:41:24.659484460Z at async exports.listen (/usr/src/app/src/webserver.js:83:2) 2023-07-04T07:41:24.659487997Z at async Object.start.start (/usr/src/app/src/start.js:45:3) 2023-07-04T07:41:24.661508745Z 2023-07-04T07:41:24.661Z [4567/300] - ESC[33mwarnESC[39m: [plugins/nodebb-plugin-write-api] Hook "filter:router.page" is deprecated, please use "response:router.page" instead. 2023-07-04T07:41:24.661884343Z 2023-07-04T07:41:24.661955056Z 2023-07-04T07:41:24.661Z [4567/300] - ESC[33mwarnESC[39m: [plugins/load] The following plugins may not be compatible with your version of NodeBB. This may cause unintended behaviour or crashing. In the event of an unresponsive NodeBB caused by this plugin, run `./nodebb reset -p PLUGINNAME` to disable it. 2023-07-04T07:41:24.662088975Z * nodebb-plugin-emoji 2023-07-04T07:41:24.662106729Z * nodebb-plugin-create-forum 2023-07-04T07:41:24.662121937Z * nodebb-plugin-telemetry-plugin 2023-07-04T07:41:24.662134421Z 2023-07-04T07:41:24.676530071Z Redis db connection {"url":"http://0.0.0.0:4567/discussions/","secret":"3768b034-c0cc-469b-9789-04c0f9d5229b","database":"redis","port":"4567","redis":{"host":"10.50.10.4","port":"6379","password":"","database":"6 "}} 2023-07-04T07:41:24.680751961Z Redis db connected. 2023-07-04T07:41:24.681096783Z 2023-07-04T07:41:24.681Z [4567/300] - ESC[32minfoESC[39m: [plugins/spam-be-gone] Settings loaded 2023-07-04T07:41:24.694823423Z 2023-07-04T07:41:24.694Z [4567/300] - ESC[32minfoESC[39m: Sunbird SSO will not be available until it is configured! 2023-07-04T07:41:24.701756176Z 2023-07-04T07:41:24.701Z [4567/300] - ESC[32minfoESC[39m: [api] Adding 0 route(s) to `api/v3/plugins` 2023-07-04T07:41:24.709947996Z 2023-07-04T07:41:24.709Z [4567/300] - ESC[32minfoESC[39m: [router] Routes added 2023-07-04T07:41:24.711751382Z 2023-07-04T07:41:24.711Z [4567/300] - ESC[32minfoESC[39m: NodeBB Ready 2023-07-04T07:41:24.713532903Z 2023-07-04T07:41:24.713Z [4567/300] - ESC[32minfoESC[39m: Enabling 'trust proxy' 2023-07-04T07:41:24.714860747Z 2023-07-04T07:41:24.714Z [4567/300] - ESC[32minfoESC[39m: NodeBB is now listening on: 0.0.0.0:4567
-
-
@sravani-64 said in Facing 504 gateway timeout error while accessing categories and tags in discussion forum:
2023-07-04T07:41:24.255252846Z 2023-07-04T07:41:24.255Z [4567/300] - ESC[33mwarnESC[39m: [plugins] Unable to load library for: nodebb-plugin-azurestorage
2023-07-04T07:41:24.255492688Z 2023-07-04T07:41:24.255Z [4567/300] - ESC[31merrorESC[39m: Error: Cannot find module './meta'Why are you using such an old version of NodeBB?
Also, in your logs, see
2023-07-04T07:41:24.255252846Z 2023-07-04T07:41:24.255Z [4567/300] - ESC[33mwarnESC[39m: [plugins] Unable to load library for: nodebb-plugin-azurestorage 2023-07-04T07:41:24.255492688Z 2023-07-04T07:41:24.255Z [4567/300] - ESC[31merrorESC[39m: Error: Cannot find module './meta'
There are two issues here. The first is that it looks as though a plugin is enabled that does not exist, you would need to use the CLI to disable that. The second could be fixed with
./nodebb build
. -
Hi @phenomlab ,
Thank you for your suggestions. We have followed the two steps- Disable nodebb-plugin-azurestorage - We were not able to reset the plugin from CLI, error was this plugin is not active on this forum, deactivated it from nodebb UI
- list item ./nodebb build is executed from CLI
After these actions, captured logs as below
2023-07-05T05:46:57.113708564Z started │ │ 2023-07-05T05:46:57.129335082Z 2023-07-05T05:46:57.129Z [4567/124] - warn: [plugins] File not found: /usr/src/app/node_modules/nodebb-plugin-create-forum/public/js/admin.js (Ignoring) │ │ 2023-07-05T05:46:57.136507560Z 2023-07-05T05:46:57.136Z [4567/124] - info: [build] Building in parallel mode │ │ 2023-07-05T05:46:57.137059895Z 2023-07-05T05:46:57.136Z [4567/124] - info: [build] plugin static dirs build started │ │ 2023-07-05T05:46:57.138879809Z 2023-07-05T05:46:57.138Z [4567/124] - info: [build] requirejs modules build started │ │ 2023-07-05T05:46:57.139824442Z 2023-07-05T05:46:57.139Z [4567/124] - info: [build] client js bundle build started │ │ 2023-07-05T05:46:57.160804666Z 2023-07-05T05:46:57.160Z [4567/124] - info: [build] admin js bundle build started │ │ 2023-07-05T05:46:57.161058154Z 2023-07-05T05:46:57.160Z [4567/124] - info: [build] client side styles build started │ │ 2023-07-05T05:46:57.161423587Z 2023-07-05T05:46:57.161Z [4567/124] - info: [build] admin control panel styles build started │ │ 2023-07-05T05:46:57.162232487Z 2023-07-05T05:46:57.162Z [4567/124] - info: [build] templates build started │ │ 2023-07-05T05:46:57.162573084Z 2023-07-05T05:46:57.162Z [4567/124] - info: [build] languages build started │ │ 2023-07-05T05:46:57.473807015Z 2023-07-05T05:46:57.473Z [4567/124] - warn: [meta/css] File not found! nodebb-plugin-create-forum/less/style.less │ │ 2023-07-05T05:46:57.521138662Z 2023-07-05T05:46:57.518Z [4567/124] - warn: [meta/css] File not found! nodebb-plugin-create-forum/less/style.less │ │ 2023-07-05T05:46:59.835347700Z 2023-07-05T05:46:59.835Z [4567/124] - info: [build] plugin static dirs build completed in 2.698sec │ │ 2023-07-05T05:47:03.841818127Z Browserslist: caniuse-lite is outdated. Please run: │ │ 2023-07-05T05:47:03.841853744Z npx browserslist@latest --update-db │ │ 2023-07-05T05:47:03.841859044Z Why you should do it regularly: https://github.com/browserslist/browserslist#browsers-data-updating 023-07-05T05:47:04.536998940Z Browserslist: caniuse-lite is outdated. Please run: │ │ 2023-07-05T05:47:04.537031822Z npx browserslist@latest --update-db │ │ 2023-07-05T05:47:04.537037743Z Why you should do it regularly: https://github.com/browserslist/browserslist#browsers-data-updating │ │ 2023-07-05T05:47:07.535159251Z 2023-07-05T05:47:07.534Z [4567/124] - info: [build] client js bundle build completed in 10.395sec │ │ 2023-07-05T05:47:07.535315042Z 2023-07-05T05:47:07.535Z [4567/124] - info: [build] admin js bundle build completed in 10.375sec │ │ 2023-07-05T05:47:07.633938283Z 2023-07-05T05:47:07.633Z [4567/124] - info: [build] client side styles build completed in 10.473sec │ │ 2023-07-05T05:47:10.044485021Z 2023-07-05T05:47:10.044Z [4567/124] - info: [build] admin control panel styles build completed in 12.883sec │ │ 2023-07-05T05:47:11.146628928Z 2023-07-05T05:47:11.146Z [4567/124] - info: [build] languages build completed in 13.984sec │ │ 2023-07-05T05:47:11.473301485Z 2023-07-05T05:47:11.473Z [4567/124] - info: [build] templates build completed in 14.311sec │ │ 2023-07-05T05:47:12.408270233Z 2023-07-05T05:47:12.408Z [4567/124] - info: [build] requirejs modules build completed in 15.27sec │ │ 2023-07-05T05:47:12.409218331Z 2023-07-05T05:47:12.409Z [4567/124] - info: [build] Asset compilation successful. Completed in 15.272sec. │ │ 2023-07-05T05:47:12.413905761Z [cluster] Restarting... │ │ 2023-07-05T05:47:12.414561070Z 2023-07-05T05:47:12.414Z [4567/124] - info: [app] Shutdown (SIGTERM/SIGINT) Initialised. │ │ 2023-07-05T05:47:12.415904020Z Clustering enabled: Spinning up 1 process(es). │ │ 2023-07-05T05:47:12.415921453Z │ │ 2023-07-05T05:47:12.416897454Z 2023-07-05T05:47:12.416Z [4567/124] - info: [app] Web server closed to connections. │ │ 2023-07-05T05:47:12.417230348Z 2023-07-05T05:47:12.416Z [4567/124] - info: [app] Live analytics saved. │ │ 2023-07-05T05:47:12.417954926Z 2023-07-05T05:47:12.417Z [4567/124] - info: [app] Database connection closed. │ │ 2023-07-05T05:47:12.418111149Z 2023-07-05T05:47:12.417Z [4567/124] - info: [app] Shutdown complete. │ │ 2023-07-05T05:47:12.441592433Z [cluster] Child Process (124) has exited (code: 0, signal: null) │ │ 2023-07-05T05:47:12.605083090Z 2023-07-05T05:47:12.604Z [4567/233] - info: Initializing NodeBB v1.18.6 http://0.0.0.0:4567/discussions/ │ │ 2023-07-05T05:47:13.685865510Z 2023-07-05T05:47:13.685Z [4567/233] - info: [socket.io] Restricting access to origin: http://0.0.0.0:* │ │ 2023-07-05T05:47:13.876491612Z nodebb plugin sunbird oidc working │ │ 2023-07-05T05:47:13.879450530Z 2023-07-05T05:47:13.879Z [4567/233] - warn: [plugins] File not found: /usr/src/app/node_modules/nodebb-plugin-create-forum/public/js/admin.js (Ignoring) │ │ 2023-07-05T05:47:14.180914073Z 2023-07-05T05:47:14.180Z [4567/233] - warn: [plugins/nodebb-plugin-telemetry-plugin] The plugin.json field "library" is deprecated. Please use the package.json field │ │ "main" instead. │ │ 2023-07-05T05:47:14.181647339Z 2023-07-05T05:47:14.181Z [4567/233] - warn: [plugins] Unable to load library for: nodebb-plugin-telemetry-plugin │ │ 2023-07-05T05:47:14.181706429Z 2023-07-05T05:47:14.181Z [4567/233] - error: Error: Cannot find module './user' │ │ 2023-07-05T05:47:14.181714294Z Require stack: │ │ 2023-07-05T05:47:14.181718642Z - /usr/src/app/src/plugins/index.js │ │ 2023-07-05T05:47:14.181722560Z - /usr/src/app/src/meta/configs.js │ │ 2023-07-05T05:47:14.181726597Z - /usr/src/app/src/meta/index.js │ │ 2023-07-05T05:47:14.181730234Z - /usr/src/app/src/start.js │ │ 2023-07-05T05:47:14.181733861Z - /usr/src/app/require-main.js 2023-07-05T05:47:14.181737688Z - /usr/src/app/app.js │ │ 2023-07-05T05:47:14.181741395Z at Function.Module._resolveFilename (node:internal/modules/cjs/loader:933:15) │ │ 2023-07-05T05:47:14.181745162Z at Function.Module._load (node:internal/modules/cjs/loader:778:27) │ │ 2023-07-05T05:47:14.181748919Z at Module.require (node:internal/modules/cjs/loader:1005:19) │ │ 2023-07-05T05:47:14.181752977Z at Object.<anonymous> (/usr/src/app/node_modules/nodebb-plugin-telemetry-plugin/library.js:1:26) │ │ 2023-07-05T05:47:14.181757194Z at Module._compile (node:internal/modules/cjs/loader:1103:14) │ │ 2023-07-05T05:47:14.181760581Z at Object.Module._extensions..js (node:internal/modules/cjs/loader:1155:10) │ │ 2023-07-05T05:47:14.181764228Z at Module.load (node:internal/modules/cjs/loader:981:32) │ │ 2023-07-05T05:47:14.181768255Z at Function.Module._load (node:internal/modules/cjs/loader:822:12) │ │ 2023-07-05T05:47:14.181772022Z at Module.require (node:internal/modules/cjs/loader:1005:19) │ │ 2023-07-05T05:47:14.181775860Z at require (node:internal/modules/cjs/helpers:102:18) │ │ 2023-07-05T05:47:14.181779747Z at Object.Plugins.requireLibrary (/usr/src/app/src/plugins/index.js:62:39) │ │ 2023-07-05T05:47:14.181783514Z at registerHooks (/usr/src/app/src/plugins/load.js:160:13) │ │ 2023-07-05T05:47:14.181787181Z at Plugins.loadPlugin (/usr/src/app/src/plugins/load.js:123:4) │ │ 2023-07-05T05:47:14.181790817Z at async Plugins.reload (/usr/src/app/src/plugins/index.js:112:3) │ │ 2023-07-05T05:47:14.181795016Z at async Plugins.init (/usr/src/app/src/plugins/index.js:85:2) │ │ 2023-07-05T05:47:14.181798662Z at async initializeNodeBB (/usr/src/app/src/webserver.js:99:2) │ │ 2023-07-05T05:47:14.181802349Z at async exports.listen (/usr/src/app/src/webserver.js:83:2) │ │ 2023-07-05T05:47:14.181806226Z at async Object.start.start (/usr/src/app/src/start.js:45:3) │ │ 2023-07-05T05:47:14.183956841Z 2023-07-05T05:47:14.183Z [4567/233] - warn: [plugins/nodebb-plugin-write-api] Hook "filter:router.page" is deprecated, please use "response:router.page" instead. │ │ 2023-07-05T05:47:14.184497416Z │ │ 2023-07-05T05:47:14.184585160Z 2023-07-05T05:47:14.184Z [4567/233] - warn: [plugins/load] The following plugins may not be compatible with your version of NodeBB. This may cause unintended behaviour │ │ or crashing. In the event of an unresponsive NodeBB caused by this plugin, run `./nodebb reset -p PLUGINNAME` to disable it. │ │ 2023-07-05T05:47:14.184766117Z * nodebb-plugin-emoji │ │ 2023-07-05T05:47:14.184777178Z * nodebb-plugin-create-forum │ │ 2023-07-05T05:47:14.184781556Z * nodebb-plugin-telemetry-plugin │ │ 2023-07-05T05:47:14.184785754Z │ │ 2023-07-05T05:47:14.198469284Z Redis db connection {"url":"http://0.0.0.0:4567/discussions/","secret":"3768b034-c0cc-469b-9789-04c0f9d5229b","database":"redis","port":"4567","redis":{"host":"10.50.10 │ │ .4","port":"6379","password":"","database":"6 "}} │ │ 2023-07-05T05:47:14.202436593Z 2023-07-05T05:47:14.202Z [4567/233] - info: [plugins/spam-be-gone] Settings loaded │ │ 2023-07-05T05:47:14.216121988Z 2023-07-05T05:47:14.216Z [4567/233] - info: Sunbird SSO will not be available until it is configured! │ │ 2023-07-05T05:47:14.217608125Z Redis db connected. │ │ 2023-07-05T05:47:14.223800830Z 2023-07-05T05:47:14.223Z [4567/233] - info: [api] Adding 0 route(s) to `api/v3/plugins` │ │ 2023-07-05T05:47:14.231608549Z 2023-07-05T05:47:14.231Z [4567/233] - info: [router] Routes added │ │ 2023-07-05T05:47:14.233499295Z 2023-07-05T05:47:14.233Z [4567/233] - info: NodeBB Ready │ │ 2023-07-05T05:47:14.235610463Z 2023-07-05T05:47:14.235Z [4567/233] - info: Enabling 'trust proxy' │ │ 2023-07-05T05:47:14.237185399Z 2023-07-05T05:47:14.237Z [4567/233] - info: NodeBB is now listening on: 0.0.0.0:4567
-
@sravani-64 said in Facing 504 gateway timeout error while accessing categories and tags in discussion forum:
nodebb-plugin-telemetry-plugin
Can you also disable this plugin? This seems to be the issue with the errors being thrown.
-
Did you just set this NodeBB up from scratch? If so, you should really consider using the newest version. The version you installed is many years old.
Also looks like you only provided startup logs. We need to see logs while those problematic requests are happening.
-
Issue is resolved, thank you @phenomlab and @PitaJ
-
@sravani-64 Can you advise what the fix was? Could be beneficial for someone else who stumbles across this thread.