Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Fail2ban may provide a false sense of security - Page 2
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Fail2ban may provide a false sense of security

2

Comments

  • Here's the cron script if anyone's interested.

    #!/bin/bash
    DATADIR='/root/scripts/ipset/data'
    LOGFILE=/var/log/fail2ban.log
    LINEFILE=${DATADIR}/lastline
    LASTLINE=$(wc -l $LOGFILE | awk '{print $1}')
    
    STARTLINE=0
    if [ -e "$LINEFILE" ]; then
        STARTLINE=$(cat $LINEFILE)
    fi
    # Allow for rotation of the log.
    if [ "$STARTLINE" -gt "$LASTLINE" ]; then
        STARTLINE=0
    fi
    
    DATA=$(/usr/bin/tail -n +${STARTLINE} $LOGFILE | grep Found | awk '{print $8}')
    
    echo $LASTLINE > $LINEFILE
    
    if [ -z "$DATA" ]; then
        exit 0
    fi
    
    DATA_ARRAY=($DATA)
    for IP in ${DATA_ARRAY[@]}; do
        if [[ $IP =~ ^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
            /usr/sbin/ipset -q test whitelist $IP
            if [ "$?" -eq "0" ]; then
                echo $IP >> ${DATADIR}/foundwhite
            else
                /usr/sbin/ipset -q add ipBlack $IP
                if [ "$?" -eq "0" ]; then
                    echo $IP >> ${DATADIR}/autoblack
                fi
            fi
        fi
    done
    
    exit 0
    
  • Create a VPS that you use as VPN, on the main VPS/Dedicated only allow logins using the IP of the secondary VPS.

  • sidewindersidewinder Member
    edited August 2018

    fail2ban is pretty overrated imo. There is no way to permaban an ip, I always spend about an hour setting it up and configuring it for every new server - it doesn't matter what operating system it is either - there is always some tweaking to get it working right.

    Recidive won't block forever - wtf?

    I'd probably bet 20% of the people that have it configured did it wrong and it's actually doing nothing.

    I ran BFD for around 5 years and there were thousands of entries in my deny file - no idea if this is a bad idea but the log files were way quieter than with fail2ban

  • @sidewinder said:
    fail2ban is pretty overrated imo. There is no way to permaban an ip, I always spend about an hour setting it up and configuring it for every new server - it doesn't matter what operating system it is either - there is always some tweaking to get it working right.

    I actually generate a blacklist file and iterate it in a script and add it to iptables. But yeah it's getting messy there :D

  • Ole_JuulOle_Juul Member
    edited August 2018

    It seems to me that this is not a case where generalizations are useful. I use fail2ban and no root login. The latter takes care of 99.99 percent it seems like since they all try to guess the root password which would be useless for login. I tend to use 12 digit passwords and I set fail2ban to 10 days mostly. How long would it take somebody using this so-called "slow" method? They would have 2 guesses per 10 days to figure out a user name, then when they got that they'd have another 2 guesses per 10 days to get a 12 digit password. If they got that far, they'd then have 2 guesses per 10 days to guess the root password. You do the math*.

    That said, I'm under no illusion that there isn't some other way in which I have yet to learn about. On my personal mail relay I've set fail2ban to allow 2 tries as well, and of course the same 10 day ban. In that case they would only have to guess the user names, which would not actually be that easy since there are very few of them.

    *Yes I'm aware that an attacker will keep changing IP, but it seems in practice that they have limited amounts of those, and 2 tries is still not enough under those circumstances, since they seem to have their own delay as well.

  • Consider these dovecot auth logs entries (which are real except for the 'domain.com'):

    Aug 09 08:47:02 auth-worker(27406): Info: sql([email protected],200.23.231.147): Password mismatch
    Aug 09 08:55:21 auth-worker(27821): Info: sql([email protected],200.152.107.102): Password mismatch
    Aug 09 09:35:16 auth-worker(30377): Info: sql([email protected],191.53.215.147): Password mismatch
    Aug 09 09:44:47 auth-worker(30573): Info: sql([email protected],187.111.58.219): Password mismatch
    
    

    Do they represent 4 random hackers, or are they conceivably part of a coordinated effort?

  • h2oh2o Member
    edited August 2018

    Hey you guys, who have the honey pot to take a look what they'r going to do after login?

    I am very curious. XD

  • HxxxHxxx Member

    Alright Sir, we will help you with your math. Is over 9000 according to my 500IQ.

    @Ole_Juul said:
    It seems to me that this is not a case where generalizations are useful. I use fail2ban and no root login. The latter takes care of 99.99 percent it seems like since they all try to guess the root password which would be useless for login. I tend to use 12 digit passwords and I set fail2ban to 10 days mostly. How long would it take somebody using this so-called "slow" method? They would have 2 guesses per 10 days to figure out a user name, then when they got that they'd have another 2 guesses per 10 days to get a 12 digit password. If they got that far, they'd then have 2 guesses per 10 days to guess the root password. You do the math*.

    That said, I'm under no illusion that there isn't some other way in which I have yet to learn about. On my personal mail relay I've set fail2ban to allow 2 tries as well, and of course the same 10 day ban. In that case they would only have to guess the user names, which would not actually be that easy since there are very few of them.

    *Yes I'm aware that an attacker will keep changing IP, but it seems in practice that they have limited amounts of those, and 2 tries is still not enough under those circumstances, since they seem to have their own delay as well.

  • Well this has worked out nicely I think.

    Changes:

    • I moved to blocking /24s rather than IPs. Wanted to see results in my lifetime.

    Blocking:

    • I'm currently blocking 1,048 /24s outside Canada/US, with 0-3 new ones being added per hour. At the start it was 15-20 per hour.
    • That's less than 0.01% of the internet blocked (please correct me).

    Affect on my users:

    • No blocking is done in Canada/US.
    • I estimate maybe 1% of my users travel outside Canada/US.
    • Abroad, users have a 1/10,000 chance worldwide of being on a blocked /24. Higher in some countries that shall go unnamed.
    • Port 443 is not blocked. Users can still access webmail even if their laptop/phone is blocked. (Repeated webmail login failures are dealt with separately - 3 strikes your out for a short time).

    Collateral damage:

    • Port 25 is not blocked. If there's a legitimate mailserver in a blocked /24 it can still send mail on port 25 to our server.

    Feedback appreciated!

  • But why can't it be forever? I don't get it...

    @willie said:
    The "recidive" (Iirc they spell it like that) rule implements longer bans and it was added a few versions ago. The version of fail2ban in the debian 9 (but not earlier) repo has it.

  • sidewinder said: But why can't it be forever? I don't get it...

    IP addresses come and go... it doesn't let that many queries through.

  • Sounds great, how are you blocking /24s that aren't in US or Canada? I like this strategy, just don't get how you can do it automagically and on the fly...

    @sleddog said:
    Well this has worked out nicely I think.

    Changes:

    • I moved to blocking /24s rather than IPs. Wanted to see results in my lifetime.

    Blocking:

    • I'm currently blocking 1,048 /24s outside Canada/US, with 0-3 new ones being added per hour. At the start it was 15-20 per hour.
    • That's less than 0.01% of the internet blocked (please correct me).

    Affect on my users:

    • No blocking is done in Canada/US.
    • I estimate maybe 1% of my users travel outside Canada/US.
    • Abroad, users have a 1/10,000 chance worldwide of being on a blocked /24. Higher in some countries that shall go unnamed.
    • Port 443 is not blocked. Users can still access webmail even if their laptop/phone is blocked. (Repeated webmail login failures are dealt with separately - 3 strikes your out for a short time).

    Collateral damage:

    • Port 25 is not blocked. If there's a legitimate mailserver in a blocked /24 it can still send mail on port 25 to our server.

    Feedback appreciated!

  • SplitIceSplitIce Member, Host Rep

    The problem with fail2ban and similar common security tools is often not the tool (which is reasonably well developed, and does what it says) but the user. A large percentage of users install these tools in a belief that doing so makes them secure (as if 100% secure is possible), it doesn't.

    Whats worse is this can lead to issues later on (something we see reasonably often) when incorrect IPs get banned (made even better by some users who don't even know they installed fail2ban/csf etc). A common cause of this is CSF more so than fail2ban due to bans enacted on udp ports or on unallocated ports (aka port scan protection) both of which are vulnerable to bans on spoofed addresses (targetting infrastructure, large clients or even other players on games that disclose player ips).

  • Jona4sJona4s Member
    edited August 2018

    If you close all your ports (-j DROP), and only connect to SSH using Port Knocking, nobody will scan your box anymore, because all ports will timeout.

    The only port that needs to be open is 80, and you have Cloudflare for that.

  • sidewinder said: Sounds great, how are you blocking /24s that aren't in US or Canada? I like this strategy, just don't get how you can do it automagically and on the fly...

    I created an ipset called 'whitelist' and then downloaded & imported CIDR-format files from ipdeny.com

    [prompt:] wget http://www.ipdeny.com/ipblocks/data/aggregated/ca-aggregated.zone
    [prompt:] wget http://www.ipdeny.com/ipblocks/data/aggregated/us-aggregated.zone
    [prompt:] ipset create whitelist hash:net
    [prompt:] ./loadwhite.sh ca-aggregated.zone
    [prompt:] ./loadwhite.sh us-aggregated.zone
    

    The loadwhite.sh script:

    #!/bin/bash
    if [ -z "$1" ]; then
            echo "Usage: ./loadwhite.sh "
            exit 0
    fi
    FILE=$1
    if [ -e "$FILE" ]; then
            while read LINE
            do
                    if [ ! -z "$LINE" ]; then
                            IP=$LINE
                            ipset add whitelist $IP
                    fi
            done < $FILE
    fi
    exit 0

    The you can test an IP against the whitelist.

    Thanked by 2plumberg sidewinder
  • mfsmfs Banned, Member

    A couple of suggestions on this ipset approach: rather than adding each ip with a loop (which is painfully painful for big sets, even more for sets meant to be updated on a daily or weekly basis) you may want to create an ipset restore list to import, and before that you may want to optimize its performance; both goals may be accomplished with iprange's print-prefix, ipset-reduce and ipset-reduce-entries switches.

    Thanked by 1sleddog
  • Fun stats! Failed authentications for smtp/imap/pop are now down to barely a handful per day.

    Here's the number of unique /24's blocked during the past week.

                               China   331  26%
                              Brazil   166  13%
                             Vietnam   122  10%
                  Russian Federation    55   4%
                             Unknown    36   3%
                               India    35   3%
                             Ecuador    30   2%
                               Egypt    29   2%
                           Indonesia    29   2%
                              Poland    27   2%
                               Korea    26   2%
                               Italy    25   2%
                            Malaysia    23   2%
                            Thailand    20   2%
                              Taiwan    16   1%
                              Mexico    15   1%
                           Argentina    13   1%
                      Czech Republic    12   1%
                               Spain    12   1%
                            Pakistan    11   1%
                            Colombia    10   1%
                             Ukraine    10   1%
                        South Africa    10   1%
                 Antigua and Barbuda     9   1%
                            Bulgaria     9   1%
                          Bangladesh     9   1%
                         Netherlands     8   1%
                               Japan     7   1%
                              Serbia     7   1%
                            Cambodia     7   1%
                         New Zealand     6   0%
                           Singapore     6   0%
                               Ghana     6   0%
                      United Kingdom     6   0%
                         Philippines     6   0%
                              Zambia     6   0%
                                Oman     5   0%
                             Belarus     5   0%
                              Belize     5   0%
                            Portugal     4   0%
                             Albania     4   0%
                               Nepal     4   0%
                             Romania     4   0%
    Lao People's Democratic Republic     4   0%
                             Georgia     3   0%
                              Turkey     3   0%
                             Lebanon     3   0%
                              Jordan     3   0%
                              Panama     3   0%
                             Hungary     3   0%
                               Kenya     3   0%
                              France     3   0%
                          Azerbaijan     3   0%
                               Chile     3   0%
                              Israel     3   0%
                  Dominican Republic     3   0%
                             Bahrain     2   0%
                            Tanzania     2   0%
                       United States     2   0%
                          Kazakhstan     2   0%
                             Moldova     2   0%
                             Germany     2   0%
                           Lithuania     2   0%
                           Australia     2   0%
                   Brunei Darussalam     2   0%
                                Iran     2   0%
                           Hong Kong     2   0%
                            Slovenia     2   0%
                United Arab Emirates     2   0%
                          Tajikistan     1   0%
                                Peru     1   0%
                           Venezuela     1   0%
                                Iraq     1   0%
                               Niger     1   0%
                             Somalia     1   0%
                             Bolivia     1   0%
                             Morocco     1   0%
                          Uzbekistan     1   0%
                          Costa Rica     1   0%
                             Belgium     1   0%
              Bosnia and Herzegovina     1   0%
                               Qatar     1   0%
                              Greece     1   0%
                  Satellite Provider     1   0%
                             Myanmar     1   0%
                            Paraguay     1   0%
                             Denmark     1   0%
                            Slovakia     1   0%
                           Macedonia     1   0%
    
                               TOTAL  1272
    
  • edited August 2018

    It's almost useless now a days imo. Only gets very low hanging fruit. Most bots spread scans out over hundreds/thousands of IP's with a large amount of time between hits on the same IP. So then you gotta increase your scan time which increases the likelyhood of false positives and banning legitimate stuff.

    It's almost not worth it for the amount of administrative overhead it adds fiddling around with it. I just use the SSH filters. Binary installs are usually configured to do that by default so no real setup required.

    I suppose it's useful to reduce noise/logging activity but not much else.

  • LosPollosHermanos said: Most bots spread scans out over hundreds/thousands of IP's with a large amount of time between hits on the same IP

    You haven't read the thread?

  • edited August 2018

    @eva2000 said:-------

    The script kiddies are about 10 steps ahead of you. They know all about fail2ban and exactly how to get around it. So you are just stopping the lowest hanging fruit by fiddling with scan/ban times etc and trying to optimize filters etc. It's a never ending thing once you start doing that.

  • edited August 2018

    @sleddog said:

    LosPollosHermanos said: Most bots spread scans out over hundreds/thousands of IP's with a large amount of time between hits on the same IP

    You haven't read the thread?

    If you don't want opinions why are you starting threads?

  • LosPollosHermanos said: So you are just stopping the lowest hanging fruit by fiddling with scan/ban times etc and trying to optimize filters etc.

    That's not what I'm doing. But I accept your input, thanks :)

  • edited August 2018

    @sleddog said:

    LosPollosHermanos said: So you are just stopping the lowest hanging fruit by fiddling with scan/ban times etc and trying to optimize filters etc.

    That's not what I'm doing. But I accept your input, thanks :)

    What are you doing?

  • LosPollosHermanos said: What are you doing?

    That's where reading the thread comes in :)

    Thanked by 1willie
  • @sleddog said:
    Well this has worked out nicely I think.

    Changes:

    • I moved to blocking /24s rather than IPs. Wanted to see results in my lifetime.

    Blocking:

    • I'm currently blocking 1,048 /24s outside Canada/US, with 0-3 new ones being added per hour. At the start it was 15-20 per hour.
    • That's less than 0.01% of the internet blocked (please correct me).

    Affect on my users:

    • No blocking is done in Canada/US.
    • I estimate maybe 1% of my users travel outside Canada/US.
    • Abroad, users have a 1/10,000 chance worldwide of being on a blocked /24. Higher in some countries that shall go unnamed.
    • Port 443 is not blocked. Users can still access webmail even if their laptop/phone is blocked. (Repeated webmail login failures are dealt with separately - 3 strikes your out for a short time).

    Collateral damage:

    • Port 25 is not blocked. If there's a legitimate mailserver in a blocked /24 it can still send mail on port 25 to our server.

    Feedback appreciated!

    sounds great!
    would you care to share the complete set of scripts you use to accomplish this?

    TIA,
    Ewald...

  • sounds great!
    would you care to share the complete set of scripts you use to accomplish this?

    TIA,
    Ewald...

    It's all in the thread except ... I don't think he left anything out.

    -- download the whitelists, add to ipset with the loadwhite script
    -- run the other script he posted via cron job to block forever.

  • LetzienLetzien Member
    edited April 2019

    @sidewinder said:

    sounds great!
    would you care to share the complete set of scripts you use to accomplish this?

    TIA,
    Ewald...

    It's all in the thread except ... I don't think he left anything out.

    -- download the whitelists, add to ipset with the loadwhite script
    -- run the other script he posted via cron job to block forever.

    Then, take both the blue and red pills, and join the rave.

  • beamzerbeamzer Member

    @sidewinder said:
    It's all in the thread except ... I don't think he left anything out.

    -- download the whitelists, add to ipset with the loadwhite script
    -- run the other script he posted via cron job to block forever.

    You're right, most of it is there. just need to set-up the blacklist as well

    ipset create ipBlack hash:ip

    and would need to change this part

    if [[ $IP =~ ^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
      ...
     /usr/sbin/ipset -q add ipBlack $IP
    

    a little if i wanted to block on /24 subnets as @sleddog mentioned
    (and change the ipset ipBlack to hash:net)

    There are a couple of ip's where maxminds geolocation seems to differ from the
    ip-deny zone's, but that is to be expected i guess.

  • Why not use ssh keys and clear the logs periodically? Or change the ssh port/disable root login. I found that these script kiddies tend to try to login to root or names like admin.

  • beamzerbeamzer Member

    @smallbibi said:
    Why not use ssh keys and clear the logs periodically? Or change the ssh port/disable root login. I found that these script kiddies tend to try to login to root or names like admin.

    Sure, and the best option is to combine security features. In my case i also run an IMAPS server, authenticated SMTP and webserver logins. So there is more to protect than ssh.
    Some of the attacks on the IMAPS server are very stealthy, less than one try per hour, so that's where a tool like this would come in handy.

Sign In or Register to comment.