@sampo2910 Okay, from the screenshot posted above, you are relaying NodeBB email through a dedicated SMTP executable. If whatever that is, let us assume Postfix, the SMTPD (SMTP Daemon) should be logging attempted connections, typically to /var/log/maillog.
Presuming you have access to the box where that daemon is running, tail that puppy:
tail -f /var/log/maillog
Then repeat whatever you do in NodeBB to trigger the NodeBB log smtp connection refused error. Observe the tail'd smtp daemon log. It should provide more detailed insight as to why the connection is being rejected. If not, increase "verbosity" of the postfix sendmail daemon and reload the config. Iirc you can even do from command line temporarily w/o having to edit the conf file but I forget the command off the top of my head. I think man postconf should get you there.
Good luck o/
P.S.; The above is predicated on you having selected Custom and then obfuscating your actual config via the smtp.mail.com example in your screenshot above. If not, see @PitaJ comment above.