@Gaurav-Grv-Robinson It looks like NodeBB uses the nodemailer package under the covers, but the UI may only present SMTP transport options.
Your best bet is to use an SMTP server that's configured to listen/login/relay email on your behalf, this is a little different than a mail spooling system that in the end may relay, or otherwise send directly. If you have an inbound mail server that also has smtp for outbound, use that.
I tend to just use SendGrid (hosted SaaS mail delivery) for my outbound emails, because delivery rates come out much better. There are other services, and depending on where you are hosting, they may offer one.