After hosting a mail server for a few years, I've gotten tired of seeing alllll the 404 attempts in my daily logwatch. Fail2Ban can help here really well, and it turns out to be really easy.
nano /etc/fail2ban/filter.d/nginx-4xx.conf (I'm counting on your running Debian and having things in default locations here), and enter the following:
[Definition] failregex = ^<HOST>.*"(GET|POST).*" (404|444|403|400) .*$ ignoreregex =
Note: Make sure you use a capital 'D' in Definition there.
Now, edit your /etc/fail2ban/jail.conf, and add these lines:
[nginx-4xx] enabled = true port = http,https logpath = /var/log/nginx/access.log maxretry = 3
Of course, modify your maxretry as desired. Refresh your Fail2Ban rules with
service fail2ban restart, and it should now be on the lookout for repeated 4xx errors.
You can confirm it's working with:
fail2ban-client status nginx-4xx:
Status for the jail: nginx-4xx |- Filter | |- Currently failed: 2 | |- Total failed: 9 | `- File list: /var/log/nginx/access.log `- Actions |- Currently banned: 2 |- Total banned: 2 `- Banned IP list: 220.127.116.11 18.104.22.168
Worth mentioning... for the first while I thought it wasn't working, but I realised I was trying to trigger the rule from on the same LAN, and ...
[DEFAULT] # time is in seconds. 3600 = 1 hour, 86400 = 24 hours (1 day) findtime = 3600 bantime = 3600 maxretry = 5 ignoreip = 127.0.0.1 127.0.0.0/8 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16
... I was hitting it from within the ignoreip range. 😬
And of course, I can't finish off a blog without giving credit where it's due. I got this all from GitHub user AysadKozanoglu, here: https://gist.github.com/AysadKozanoglu/1335735272fb3b00a03bd3eea22af818