make sure you have port 80 and/or 443 open on your firewall. I see 80 is open (I get the "Welcome to Nginx" message). So now you need to make a few changes in your nginx config for it to redirect 4567 to your site on 80/443 (http/https), as currently it is set to the nginx page.
The docs has a config you can use, just make the changes like Pitaj has suggested and for https, you will need a certificate. Lets-Encrypt is free
**Edit to add: Don't forget to restart nginx to use the new config
Yes, with a VPS you can install whatever you want on it, much like your own desktop computer. Install WordPress, or Drupal, or Ghost, or whatever you want. As long as the VPS resources are enough to handle all of them running at once 😄