IP banning

[dwn]

So, you've programmatically identified a malicious request from an IP address. How do you blacklist them to prevent them from crawling/scanning/etc the rest of the site? Just notify the admin that there's a bad IP and make them go ban them by hand?

• Use iptables
  • Requires root privileges to add an entry. Bad thing to give to software.
• Add them to some blacklist in your proxy.
  • Different depending on the proxy used. Easy but high-impact in nginx(do include /opt/nodebb/blacklist.conf in nginx.conf, add deny 1.2.3.4; for each blocked user, reload nginx after each add)
    • potentially suitable to be a plugin? managing it similar to the emailer would allow other plugins to trigger IP bans in a common way regardless of the IPbanner used.
• Build in some nodebb functionality to drop their requests.
  • Wouldn't depend on the environment.
  • Is there still some performance impact to the site if they're sending many requests and nodebb is handling blocking them?

In my eyes this functionality must be provided in some way to legitimize using NodeBB for large environments. It's too tedious to ask administrators to handle banning IPs by hand.

[a_5mith]

I would always recommend banning using nginx or Apache. Making the software handle who's ip banned is an unnecessary use of resources when deny from IP is so quick and simple. Its a waste of a socket connection in this case.

In addition, banning the IP has no effect on that user being banned. I could obtain a new IP address faster than you could ban it in most cases. So banning IPs (unless you know they're static, like a corporate business and its subnets), then it's of no use implementing such a feature. You'd be far better off implementing some sort of hell banning technique that doesn't stop them, but persuades them to give up. Random 502 errors, random 404s, unresponsive link clicking. The inability to make posts due to "unknown errors" , blank pages. Etc. Annoy them enough, they'll leave. Basically.

[dwn]

@a_5mith said:

I would always recommend banning using nginx or Apache. Making the software handle who's ip banned is an unnecessary use of resources when deny from IP is so quick and simple. Its a waste of a socket connection in this case.

Is there a way to deny from IP in nginx that doesn't require a config reload? That's a pretty resource-intensive operation.

I could obtain a new IP address faster than you could ban it in most cases.
Etc. Annoy them enough, they'll leave. Basically.

So, the reasoning behind making this functionality is not so much to handle getting rid of mean forum posters who might register new accounts(the time taken to scan for ipv4 proxies and register and make a post would definitely be greater than the time taken to click a "nuke user" button on their post, though).

Automated attacks exist that can be detected very easily programmatically -- and an IP ban issued to stop them. If I have a botnet open socket requests in a specific pattern to your site, it will DoS it pretty easily with very little bandwidth used. Are you going to iptables/proxy ban each of those IPs by hand as they come? It would be a simple matter to have a plugin listen on this functionality and trigger IPbans when an obvious attack is detected.

also I'd hope whatever functionality for IPbanning would not even allow them to open a socket -- serve them some junk static html.

[a_5mith]

Reloading nginx is something that can be worked around, and only happens say, once every now and then.

Modern proxies (including free ones) are very good at getting a new IP address. Failing that, disconnecting and reconnecting would also get you a new IP address on software based proxies (VPNs etc)

Automated attacks are slightly different, it depends on the attack, if it's one user being a nerd, then I would IP ban them in nginx. If it's a ddos attack, then there's no way I could prevent this using software as by the time theyd reached the software, the connection would already be bottle necked and the best course of action at that point would be damage control. Not prevention.

[dwn]

So there's obviously something getting lost in translation here... I work in security, I know how proxies and botnets work very well.

@a_5mith said:

Reloading nginx is something that can be worked around, and only happens say, once every now and then.

Not if I've got a net of 20k coming at you. I can stagger them so that your reloads alone(if automated) interrupt service for users for a significant period of time or cause significant annoyance.

Modern proxies (including free ones) are very good at getting a new IP address. Failing that, disconnecting and reconnecting would also get you a new IP address on software based proxies (VPNs etc)

So, the modern internet has the concept of "IP Blacklists." You are not going to find a proxy quickly that is not already on one of these lists. VPNs have a very limited pool of addresses that they pull from.

Not that this matters to any remotely sophisticated attacker: their proxy lists will be from a botnet they control(still going to be a fairly limited pool), or a scan of a large swath of the ipv4 address space that they performed at an earlier time(prob < 1k working from this). The average kid you'd IP ban doesn't have access to this kind of thing, though.

Automated attacks are slightly different, it depends on the attack, if it's one user being a nerd, then I would IP ban them in nginx. If it's a ddos attack, then there's no way I could prevent this using software as by the time theyd reached the software, the connection would already be bottle necked and the best course of action at that point would be damage control. Not prevention.

So ddos is a terrible generic term. It can mean a LOT of things.

The case you described is a bandwidth based DDoS which is actually sorta rare when it comes to them(solutions like cloudflare help prevent bandwidth-based and other simple DDoS methods very well).

The type of DoS that nothing but the application can help you with will take advantage of requests that trigger slow-running or resource intensive code paths. It turns out there's a lot of these and really no external tools that can help you in a websocket-based application. This kind of DoS can be done via a single IP at a time as they don't usually take a lot of bandwidth (of course if you ban one IP it's trivial to automate having another bot continue the attack). They are not complex attacks.

You've got to look for these and stop them at the application layer. This is done via behavioral/attack pattern analysis. If you know a certain code path will easily DoS your site, you can block these attacks via code -- it's very simple. If the software doesn't give you an easy way to programmatically ignore traffic from a certain address/range, you're sort of left to ratelimit everyone(which sucks).

This is just one kind of attack. IP banning applies to a myriad of attacks and is really something necessary for the application to be able to do.

[Peter-Zoltan Keresztes]

Why don't you do that on firewall level. If you don't want to add rules manually try fail2ban

[dwn]

@Peter-Zoltan-Keresztes said:

Why don't you do that on firewall level. If you don't want to add rules manually try fail2ban

Here's a UX example: What if someone's on an IP blacklist? Timing out their connection to the site is a terrible UX compared to a static page saying they are IPbanned.

[Scuzz]

I would also like some type of IP banning and Search user by IP function.

[julian]

@Scuzz said:

I would also like some type of IP banning and Search user by IP function.

This is coming

[A Former User]

@julian said:

This is coming

Just like that, one of my plugin ideas got a lot more simple. And that, my children, is why we wait on @julian to make the magic happen.

[julian]

@Ted Thanks haha... although the kudos should go to @baris, who handled the updates to the user search field in the ACP

[PitaJ]

@dwn I would create a node.js layer on top of the NodeBB layer that detects current traffic and denies passthrough to anyone on a black list

[julian]

@dwn said:

Here's a UX example: What if someone's on an IP blacklist? Timing out their connection to the site is a terrible UX compared to a static page saying they are IPbanned.

Are you saying if the user is on an IP blacklist that I put him/her on? Then in that case, I'd say they deserve it, and I wouldn't need to put extra effort into displaying a nice HTTP 400 page.

In terms of false positives, that's an other issue altogether.

[dwn]

"IP Blacklist" as in a spam/malware blacklist. These are lists that security-conscious owners would naturally block, but occasionally contain innocent infected users.