@baris creation of index for tid can help in this case? can it harm something?
what additional indexes would you suggest to add?

@julian @PitaJ It looks like when I am logged in it selects additional not indexes tables. We need to improve, make faster queries of the newly added topics. Can you suggest more indexes exactly for logged in user?

@julian When disabling the plugin and rebuild the Speed is the same. But day by day of just waiting it becomes quicker since the last creation day of topics

@pitaj But when I log out the speed is quick as usual. Could it be related to the indexes with such a behavior?

@pitaj I've just checked the indexes in db, and find out that nodebb db size is 0. So I forgot that I store all the data for nodebb forum in the different one db, and that db already had the indexes. So no changes were made to the db with nodebb forum
The reason of long load page time is still mystery but its related somehow to write-api plugin.
I need to use this plugin to post more topics but I need to fix something to avoid such bad situation with long load page time

Today the page load time is about 5-10 secs for logged in user (previously it was more then 1min), so maybe some sessions are destroyed or something?

I kept write-api-plugin disabled and did not use it for 2 days

@pitaj

I've disabled all the plugins - CPU is becoming lower but I still have timeouts when log in and refresh (almost all the time > 1min, sometimes 58secs)

@baris I've just tried with newly created user and it's the same situation - timeouts

@pitaj The problem appeared after I installed nodebb-write-api plugin and post several tens of thousands topics, the more topics I posted the longer time for refreshing the page with logged in user

So I do not think other plugins rather then nodebb-write-api impact this issue

@baris The problem with 504 timeouts appears only when I try to login or refresh with logged in user. Maybe some cache for sessions should be cleared or something related to user data?

@baris what tool do you advise to analyze a network usage?
I do not have my custom code only plugins

@baris
Here is a log if this can helps

2021-12-17T20:57:30.033Z [4567/18089] - info: Initializing NodeBB v1.14.1 https://.........
2021-12-17T20:57:34.506Z [4567/18089] - info: [socket.io] Restricting access to origin: https://............:*
2021-12-17T20:57:34.733Z [4567/18089] - error: [emailer] Failed to build custom email templates {"0":"E","1":"r","2":"r","3":"o","4":"r","5":":","6":" ","7":"E","8":"A","9":"C","10":"C","11":"E","12":"S","13":":","14":" ","15":"p","16":"e","17":"r","18":"m","19":"i","20":"s","21":"s","22":"i","23":"o","24":"n","25":" ","26":"d","27":"e","28":"n","29":"i","30":"e","31":"d","32":",","33":" ","34":"o","35":"p","36":"e","37":"n","38":" ","39":"'","40":"/","41":"v","42":"a","43":"r","44":"/","45":"w","46":"w","47":"w","48":"/","49":"n","50":"o","51":"d","52":"e","53":"b","54":"b","55":"/","56":"b","57":"u","58":"i","59":"l","60":"d","61":"/","62":"p","63":"u","64":"b","65":"l","66":"i","67":"c","68":"/","69":"t","70":"e","71":"m","72":"p","73":"l","74":"a","75":"t","76":"e","77":"s","78":"/","79":"e","80":"m","81":"a","82":"i","83":"l","84":"s","85":"/","86":"b","87":"a","88":"n","89":"n","90":"e","91":"d","92":".","93":"j","94":"s","95":"'"}
2021-12-17T20:57:35.292Z [4567/18089] - warn: [deprecated] requiring core modules with `module.parent.require('./database')` is deprecated. Please use `require.main.require("./src/<module_name>")` instead.
    at Object.YoutubeLite.init [as method] (/var/www/nodebb/node_modules/nodebb-plugin-youtube-embed/library.js:20:22)
2021-12-17T20:57:35.458Z [4567/18089] - error: [plugins] Error executing 'static:app.load' in plugin 'nodebb-plugin-emoji'
2021-12-17T20:57:35.459Z [4567/18089] - error: Error: EACCES: permission denied, unlink '/var/www/nodebb/node_modules/nodebb-plugin-emoji/build/emoji/aliases.json'
2021-12-17T20:57:35.477Z [4567/18089] - info: Routes added
2021-12-17T20:57:35.480Z [4567/18089] - info: NodeBB Ready
2021-12-17T20:57:35.482Z [4567/18089] - info: Enabling 'trust proxy'
2021-12-17T20:57:35.486Z [4567/18089] - error: NodeBB address in use, exiting... {"0":"E","1":"r","2":"r","3":"o","4":"r","5":":","6":" ","7":"l","8":"i","9":"s","10":"t","11":"e","12":"n","13":" ","14":"E","15":"A","16":"D","17":"D","18":"R","19":"I","20":"N","21":"U","22":"S","23":"E","24":":","25":" ","26":"a","27":"d","28":"d","29":"r","30":"e","31":"s","32":"s","33":" ","34":"a","35":"l","36":"r","37":"e","38":"a","39":"d","40":"y","41":" ","42":"i","43":"n","44":" ","45":"u","46":"s","47":"e","48":" ","49":"0","50":".","51":"0","52":".","53":"0","54":".","55":"0","56":":","57":"4","58":"5","59":"6","60":"7","61":"\n","62":" ","63":" ","64":" ","65":" ","66":"a","67":"t","68":" ","69":"S","70":"e","71":"r","72":"v","73":"e","74":"r","75":".","76":"s","77":"e","78":"t","79":"u","80":"p","81":"L","82":"i","83":"s","84":"t","85":"e","86":"n","87":"H","88":"a","89":"n","90":"d","91":"l","92":"e","93":" ","94":"[","95":"a","96":"s","97":" ","98":"_","99":"l","100":"i","101":"s","102":"t","103":"e","104":"n","105":"2","106":"]","107":" ","108":"(","109":"n","110":"e","111":"t","112":".","113":"j","114":"s","115":":","116":"1","117":"2","118":"8","119":"0","120":":","121":"1","122":"4","123":")","124":"\n","125":" ","126":" ","127":" ","128":" ","129":"a","130":"t","131":" ","132":"l","133":"i","134":"s","135":"t","136":"e","137":"n","138":"I","139":"n","140":"C","141":"l","142":"u","143":"s","144":"t","145":"e","146":"r","147":" ","148":"(","149":"n","150":"e","151":"t","152":".","153":"j","154":"s","155":":","156":"1","157":"3","158":"2","159":"8","160":":","161":"1","162":"2","163":")","164":"\n","165":" ","166":" ","167":" ","168":" ","169":"a","170":"t","171":" ","172":"d","173":"o","174":"L","175":"i","176":"s","177":"t","178":"e","179":"n","180":" ","181":"(","182":"n","183":"e","184":"t","185":".","186":"j","187":"s","188":":","189":"1","190":"4","191":"6","192":"1","193":":","194":"7","195":")","196":"\n","197":" ","198":" ","199":" ","200":" ","201":"a","202":"t","203":" ","204":"p","205":"r","206":"o","207":"c","208":"e","209":"s","210":"s","211":".","212":"_","213":"t","214":"i","215":"c","216":"k","217":"C","218":"a","219":"l","220":"l","221":"b","222":"a","223":"c","224":"k","225":" ","226":"(","227":"i","228":"n","229":"t","230":"e","231":"r","232":"n","233":"a","234":"l","235":"/","236":"p","237":"r","238":"o","239":"c","240":"e","241":"s","242":"s","243":"/","244":"n","245":"e","246":"x","247":"t","248":"_","249":"t","250":"i","251":"c","252":"k","253":".","254":"j","255":"s","256":":","257":"6","258":"3","259":":","260":"1","261":"9","262":")"}
2021-12-17T20:57:35.490Z [4567/18089] - error: uncaughtException: listen EADDRINUSE: address already in use 0.0.0.0:4567
Error: listen EADDRINUSE: address already in use 0.0.0.0:4567
    at Server.setupListenHandle [as _listen2] (net.js:1280:14)
    at listenInCluster (net.js:1328:12)
    at doListen (net.js:1461:7)
    at process._tickCallback (internal/process/next_tick.js:63:19) {"error":{"code":"EADDRINUSE","errno":"EADDRINUSE","syscall":"listen","address":"0.0.0.0","port":4567},"stack":"Error: listen EADDRINUSE: address already in use 0.0.0.0:4567\n    at Server.setupListenHandle [as _listen2] (net.js:1280:14)\n    at listenInCluster (net.js:1328:12)\n    at doListen (net.js:1461:7)\n    at process._tickCallback (internal/process/next_tick.js:63:19)","exception":true,"date":"Fri Dec 17 2021 20:57:35 GMT+0000 (Coordinated Universal Time)","process":{"pid":18089,"uid":1000,"gid":1000,"cwd":"/var/www/nodebb","execPath":"/usr/bin/node","version":"v10.22.0","argv":["/usr/bin/node","/var/www/nodebb/app.js"],"memoryUsage":{"rss":158556160,"heapTotal":124567552,"heapUsed":96504736,"external":18387256}},"os":{"loadavg":[4.8525390625,4.66357421875,4.0390625],"uptime":191949},"trace":[{"column":14,"file":"net.js","function":"Server.setupListenHandle [as _listen2]","line":1280,"method":"setupListenHandle [as _listen2]","native":false},{"column":12,"file":"net.js","function":"listenInCluster","line":1328,"method":null,"native":false},{"column":7,"file":"net.js","function":"doListen","line":1461,"method":null,"native":false},{"column":19,"file":"internal/process/next_tick.js","function":"process._tickCallback","line":63,"method":"_tickCallback","native":false}]}
2021-12-17T20:57:35.492Z [4567/18089] - error: Error: listen EADDRINUSE: address already in use 0.0.0.0:4567
    at Server.setupListenHandle [as _listen2] (net.js:1280:14)
    at listenInCluster (net.js:1328:12)
    at doListen (net.js:1461:7)
    at process._tickCallback (internal/process/next_tick.js:63:19)
2021-12-17T20:57:35.493Z [4567/18089] - info: [app] Shutdown (SIGTERM/SIGINT) Initialised.
2021-12-17T20:57:35.496Z [4567/18089] - error: Error [ERR_SERVER_NOT_RUNNING]: Server is not running.
    at Server.close (net.js:1566:12)
    at Object.onceWrapper (events.js:286:20)
    at Server.emit (events.js:203:15)
    at emitCloseNT (net.js:1619:8)
    at process._tickCallback (internal/process/next_tick.js:63:19)
[cluster] Child Process (18089) has exited (code: 1, signal: null)
[cluster] Spinning up another process...

@baris any other suggestions?

@baris No, I still have this problem

@baris

done and what next?

@baris

it looks like no indexes

@julian which collections should I check?

@julian As I know build in write api is implemented in nodebb v15 and above. But I have nodebb v14, so need to install plugin. Do you know this issue what should I do to fix? For now I cant even login by any user I have always 504 timeouts after 1min of loading

@baris

I use nodebb v14.1 as I remember and I did not do any changes in db after installing.
I do not have custom code. The problem is appeared after installed write-api and posted several ten thousand topics. So when logged in the are always timeouts for now I cant even open /admin - 504 Gateway Time-out

Because I've posted one more bunch of topics using write-api. It looks like something is here related to nodebb-write-api. The more topics I posted the longer time of page loading with logged in user I have.
@julian it looks like you are the author of nodebb-write-api plugin that I've installed and use What can be the problem here and how to fix that? Any ideas?

@baris

refreshing the page with logged in user