Nginx & Apache, with NodeBB?

[Danny McWilliams]

I expanded my current server NodeBB hosted on Digital Ocean to 2GB and 2 cores with the intention of firstly adding a mail server then later perhaps expanding the site. Apache and Nginx aren't quite working together, and I'd love some advice.

I installed iRedMail, which installed Apache, and some other stuff (RoundCube, AWStats for example).

The install went fine, and I now have a working mail server. After install, iRedMail was redirecting to mydomain.com/mail when you tried to go to our home URL, but this was simply fixed as it was an HTML redirect.

Then, NodeBB wouldn't start - I was using too much memory. Hence the upgrade, but that would have been needed eventually anyway.

The home URL (my forum), is 403ing. As I understand it, this is because both Nginx and Apache aren't configured properly.

I've read and reread a lot of different sources and found nothing which has worked. I have checked out here for proxying, but it only covers either or, not both together.

I'm not even sure it is the cause of the 403! The error page clearly shows Nginx is spitting out the 403 error, rather than Apache.

Help!

[nhl.pl]

@Danny-McWilliams said:

I'm not even sure it is the cause of the 403! The error page clearly shows Nginx is spitting out the 403 error, rather than Apache.

Could you please post a link?

Nginx can work on top of Apache.

Apache can also work without Nginx.

Nginx of course can work without Apache.

[Danny McWilliams]

@nhl-pl http://www.onlyanexcuse.com/ is where my NodeBB forum is, the problem has been since installing iRedMail (with Apache). Our webmail (roundcube) is on http://www.onlyanexcuse.com/mail, and that's coming up just fine.

[nhl.pl]

In such scenario you can configure all other scripts to work with nginx. You do not need Apache anymore or am I missing something?

You need to configure vhosts in nginx.

[Danny McWilliams]

As you posted this, I turned off Apache and the rest is unaffected. Thanks @nhl.pl, i'll have a look there and post any solution I have.

[Danny McWilliams]

Maybe there's something wrong among these.

/etc/nginx/sites-available/default

server {

server_name www.onlyanexcuse.com;

listen 80 default_server;
listen [::]:80 default_server ipv6only=on;

root /home/errapolis/nodebb;
index index.html index.htm;

# Make site accessible from http://localhost/
server_name localhost;

location / {
	# First attempt to serve request as file, then
	# as directory, then fall back to displaying a 404.
	#try_files $uri $uri/ =404;
	# Uncomment to enable naxsi on this location
	# include /etc/nginx/naxsi.rules

proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_set_header X-NginX-Proxy true;

    proxy_pass http://127.0.0.1:4567/;
    proxy_redirect off;

    # Socket.IO Support
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
}

config.json

{
"url": "http://www.onlyanexcuse.com",
"secret": "XXXX",
"database": "redis",
"redis": {
    "host": "127.0.0.1",
    "port": "6379",
    "password": "",
    "database": "0"
}

}

[Shaun]

I don't think you can have two server_name directives in the Nginx config.

server_name www.onlyanexcuse.com localhost;

should do the same thing.

Edit: Nevermind - it does seem to work.

[Danny McWilliams]

@Shaun - Edited file and restarted Nginx. Cheers, didn't see that before.

Not solved the problem, but was needing sorted regardless.

[Shaun]

@Danny-McWilliams said:

  # First attempt to serve request as file, then
  # as directory, then fall back to displaying a 404.
  #try_files $uri $uri/ =404;
  # Uncomment to enable naxsi on this location
  # include /etc/nginx/naxsi.rules

I think this section is unnecessary and probably the problem? It's trying to serve files from the file system and not just redirecting to nodebb. This is my Nginx config for just NodeBB:

server {
        listen 80;

        server_name forums.domain.example.com;

        location / {
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header Host $http_host;
                proxy_set_header X-NginX-Proxy true;

                proxy_pass http://127.0.0.1:4567/;
                proxy_redirect off;

                # Socket.IO Support
                proxy_http_version 1.1;
                proxy_set_header Upgrade $http_upgrade;
                proxy_set_header Connection "upgrade";
        }
}

To add your webmail to that, you would make another location block for location /mail {} and direct it to your webmail product.

[Shaun]

Oh, hell. I'm batting 0 today. That section is fully commented and I just can't read.

Still, I would try to replace the config with one that just serves NodeBB and then add the /mail location block to it after it's working for NodeBB.

[Shaun]

For anyone else who may end up trying to help, here:

After having a discussion with @Danny-McWilliams I was able to somewhat isolate the problem:

• Stopping Nginx stops responses on port 80, so it appears to be the responsible server.
• cUrl can retrieve the NodeBB home page from http://127.0.0.1:4567/ on his server so neither nodejs nor nodebb appear to be the problem.
• His server still serves the /mail URL (with a redirect to HTTPS) after replacing his only sites-enabled config file with the content of my example config above. He mentioned that installing his webmail product had modified the files in /etc/nginx, so I believe there is something happening to the Nginx config outside of the sites-enabled directory that is causing the problem.

[nhl.pl]

@Shaun said:

• His server still serves the /mail URL (with a redirect to HTTPS) after replacing his only sites-enabled config file with the content of my example config above. He mentioned that installing his webmail product had modified the files in /etc/nginx, so I believe there is something happening to the Nginx config outside of the sites-enabled directory that is causing the problem.

It could be /etc/nginx/sites-available/ with a symlink to it from the /etc/nginx/sites-enabled/.

[Shaun]

I probably should have been more clear - the only symlink in /etc/nginx/sites-enabled/ is to the config file I posted above (which is, of course, in sites-available, as is standard practice for Nginx). That should do nothing but redirect all requests to NodeBB, so somehow it's still picking up some config settings that are doing a redirect on /mail to HTTPS (and something is serving on port 443, obviously, because that also gets a response).

Of course, that's not the primary location of Nginx's config file, it's just included from the main config files - so anything in those files could be overriding it, and he said that whatever mail program he installed changed Nginx's files.

Might be good to have someone better with Linux than me looking at it.

He said nmap reported something named "tram" was on port 4567. I know what nmap is but I'm not actually familiar with using it, so I don't know where it gets that information and as I said, I had him cUrl to the loopback address on that port and he reported getting the HTML for the NodeBB template.

[nhl.pl]

I think I've found it.

[Danny McWilliams]

Many thanks for your help last night @Shaun it's narrowed it down somewhat.

@nhl.pl that's exactly the problem I have right now.

Now that i'm armed with coffee and some spare time, I'm going to get a look and see if anything stands out.

[Danny McWilliams]

My nginx default.conf:

upstream php_workers {
server unix:/var/run/php-fpm.socket;
}

# HTTP
server {
listen 80;
server_name _;

root /var/www/html;
index index.php index.html index.htm;

location / {
    root /var/www/html;
}

# Normal PHP scripts
location ~ \.php$ {
    include fastcgi_params;
    fastcgi_pass php_workers;
}

# Redirect webmail/SOGo/iredadmin to HTTPS
location ~ ^/mail { rewrite ^ https://$host$request_uri?; }
location ~* ^/sogo { rewrite ^ https://$host/SOGo; }
location ~ ^/iredadmin { rewrite ^ https://$host$request_uri?; }

# Deny all attempts to access hidden files such as .htaccess.
location ~ /\. {
    deny all;
    access_log off;
    log_not_found off;
}

# Handling noisy favicon.ico messages
location = /favicon.ico {
    access_log off;
    log_not_found off;
}
}

# HTTPS
server {
listen 443;
server_name _;

ssl on;
ssl_certificate /etc/ssl/certs/iRedMail.crt;
ssl_certificate_key /etc/ssl/private/iRedMail.key;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

index index.php index.html index.htm;

location / {
    root /var/www/html;
}

# Deny all attempts to access hidden files such as .htaccess.
location ~ /\. {
    deny all;
    access_log off;
    log_not_found off;
}

# Handling noisy favicon.ico messages
location = ^/favicon.ico {
    access_log off;
    log_not_found off;
}

# Roundcube webmail
location ~ ^/mail(.*)\.php$ {
    include fastcgi_params;
    fastcgi_pass php_workers;
    fastcgi_index index.php;
    fastcgi_param SCRIPT_FILENAME /usr/share/apache2/roundcubemail$1.php;
}

location ~ ^/mail(.*) {
    alias /usr/share/apache2/roundcubemail$1;
    index index.php;
}

location ~ ^/mail/(bin|SQL|README|INSTALL|LICENSE|CHANGELOG|UPGRADING)$ { deny all; }

# Normal PHP scripts
location ~ \.php$ {
    include fastcgi_params;
    fastcgi_pass php_workers;
}

# iRedAdmin: static files under /iredadmin/static
location ~ ^/iredadmin/static/(.*)\.(png|jpg|gif|css|js) {
    alias /usr/share/apache2/iredadmin/static/$1.$2;
}

# iRedAdmin: Python scripts
location ~ ^/iredadmin(.*) {
    rewrite ^/iredadmin(/.*)$ $1 break;
    include uwsgi_params;
    uwsgi_pass unix:/var/run/uwsgi_iredadmin.socket;
    uwsgi_param UWSGI_CHDIR /usr/share/apache2/iredadmin;
    uwsgi_param UWSGI_SCRIPT iredadmin;
    uwsgi_param SCRIPT_NAME /iredadmin;
}
# iRedAdmin: redirect /iredadmin to /iredadmin/
location = /iredadmin {
    rewrite ^ /iredadmin/;
}

# SOGo
location ~ ^/sogo { rewrite ^ https://$host/SOGo; }
location ~ ^/SOGO { rewrite ^ https://$host/SOGo; }

# For IOS 7
location = /principals/ {
    rewrite ^ https://$server_name/SOGo/dav;
    allow all;
}

location ^~ /SOGo {
    proxy_pass http://127.0.0.1:20000;
    #proxy_redirect http://127.0.0.1:20000/SOGo/ /SOGo;
    # forward user's IP address
    #proxy_set_header X-Real-IP $remote_addr;
    #proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    #proxy_set_header Host $host;
    proxy_set_header x-webobjects-server-protocol HTTP/1.0;
    #proxy_set_header x-webobjects-remote-host 127.0.0.1;
    #proxy_set_header x-webobjects-server-name $server_name;
    #proxy_set_header x-webobjects-server-url $scheme://$host;
}

location ^~ /Microsoft-Server-ActiveSync {
    proxy_pass http://127.0.0.1:20000/SOGo/Microsoft-Server-ActiveSync;
    proxy_redirect http://127.0.0.1:20000/Microsoft-Server-ActiveSync /;
}

location ^~ /SOGo/Microsoft-Server-ActiveSync {
    proxy_pass http://127.0.0.1:20000/SOGo/Microsoft-Server-ActiveSync;
    proxy_redirect http://127.0.0.1:20000/SOGo/Microsoft-Server-ActiveSync /;
}

location /SOGo.woa/WebServerResources/ {
    alias /usr/lib/GNUstep/SOGo/WebServerResources/;
}
location /SOGo/WebServerResources/ {
    alias /usr/lib/GNUstep/SOGo/WebServerResources/;
}
location ^/SOGo/so/ControlPanel/Products/([^/]*)/Resources/(.*)$ {
    alias /usr/lib/GNUstep/SOGo/$1.SOGo/Resources/$2;
}

}

[Danny McWilliams]

iRedMail thankfully left a backup copy of nginx.conf and I renamed the "new" one and put the copy back on.

As a result, the forum is working again - as is my mail.

I have absolutely no idea how it broke it, but that was my solution.

I don't know if this makes anything less secure or anything, but at least it's working for now.

[Shaun]

No problem - glad I was able to help you locate the problem.

As long as your SSL certificate is still in place and the iRedMail functionality is all working, I don't think there should be an impact on the security of your system.