By default, NodeBB instances should be able to send email out of the box. However, having the server send emails is not ideal, as the server itself does not have any reputation (and in some cases, may have a negative reputation if a spammer sent messages from the IP in the past).
In nearly all cases, you will want to open up an account with a third-party emailer service, who can handle these outgoing messages for you.
Here at NodeBB, we recommend "nodebb-plugin-emailer-sendgrid", which is installable via the admin control panel. SendGrid has a free tier that is fairly generous, so you should not have to pay any upfront or recurring costs.
Once the plugin is installed and activated, you will need to register a new account at SendGrid and create an API key for NodeBB to use.