Ahh, thanks. Okay, that makes sense now. It sounds like it's failing because of the special port. Since that port is sending directly to NodeBB and not through the Apache proxy, NodeBB does not know the SSL key, so it will fail the connection.
I think there used to be a way to feed the SSL Key path to NodeBB in config.json, but I don't see any docs on it anymore, and I'm not even sure if that would work.
I think your best bet is to try getting websockets to work through Apache2, there seems to be several methods even though there's no official support.