Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Weird DoS attempt
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Weird DoS attempt

xyzxyz Member
edited June 2014 in Help

For the past few days, I've been getting a flood of HTTP requests at times of the day. However I don't understand the odd and consistent pattern of these attacks - here some facts:

  • There's exactly 2 attack sessions every day
  • Each session lasts for about 30 minutes (not exact, but usually quite close); I haven't yet seen the sessions overlap and the sessions appear to be 1-4 hours apart (ie one session will finish, then the next starts a few hours later)
  • Only a single URL is hit during each session
  • Each session comes from a new IP (have seen IPs reused); the user agent also gets randomized (probably picked from a list of common UAs) and a new URL is selected. Selected URLs seem to be random as I can't see any pattern
  • IPs are all from Ukraine and a number of them seem to be from the same network
  • The rate is around 23 reqs/sec and doesn't seem to have any adverse affects on the site, since the webserver seems to handle it fine

If the above was a bit confusing, consider this as an example:
on 2014-06-15 at 10:00, IP 1.2.3.4 starts continuously requesting /content/33, then stops at 10:30
and on the same day at 13:20, IP 5.7.8.9 starts continuously requesting /listing/8 until 13:51
Each session results in around 42,000 requests being made.

This behavior seems odd to me and I don't understand the reasoning behind it. Would anyone here have any clue?
If a DoS attack is meant to try to take down a website, why would the attack stop after half an hour? Why would there be only two attempts every day? Also, if the attack doesn't seem to be working, why would the attacker keep trying the same strategy over and over again, without making any modification?

Comments

  • Slowloris?

    RUDY?

    Seems like one of those really stupid skiddy booters, but it might be a scaper or something else that's simply intent on indexing a single page at a time [ really wouldn't surprise me, we've seen odd traffic patterns before, but not quite like the one you described. ]

    How many requests in each 30 minute period ?

  • it seems like bad bots to me.is that site popular?

  • xyzxyz Member
    edited June 2014

    GoodHosting said: How many requests in each 30 minute period ?

    Around 42,000 (so average of 23 requests/second). I don't think a scraper needs to make that many requests to a single URL...

    GoodHosting said: Slowloris?

    RUDY?

    Thanks for the pointers there. Doesn't appear to be though - the requests are all GET. I can't tell if the requests are holding connections open for a long time, but nginx here is set to 1500 max connections, so, if I understand it correctly, it would've been exhausted way before 42k requests hit?

  • linuxthefishlinuxthefish Member
    edited June 2014

    Stuff like that will kill apache in seconds, anything nginx/lighttpd will have no issue with that type traffic on static files.

    Quick and dirty solution could be http://deflate.medialayer.com/

  • xyzxyz Member
    edited June 2014

    Thanks for the tip @linuxthefish

    Using nginx here - I probably should probably make use of the limit_conn module, although concurrent connection limits don't appear to be the issue, assuming nginx logs those.

  • Been getting something similar on one of my sites, top country to visit is Ukraine.

    Its usually from random IPs as well, site has nothing to do with anything outside of the US. It does have a phpbb forum that they seem to try and register on constantly but the anti-bot thing seems to stop that.

  • said: Only a single URL is hit during each session

    What is on said URL?

  • jarjar Patron Provider, Top Host, Veteran

    Mind pasting a sample of the logs? Easier to see patterns I might recognize.

  • xyzxyz Member

    AThomasHowe said: What is on said URL?

    Nothing particularly outstanding - URLs that a typical user might browse. It's not the index/home page, and links to these pages are often buried several pages deep (ie you'd have to navigate through a few pages before you found the ones they're targeting). As the IP never requests anything other than this page, it's either found it via a search engine or someone put in a list of URLs to target (latter seems unlikely as the URLs don't seem to have much similarity, for example, one might be an article from 2012, another might be from more recent, another might be an old listing page etc).

    I don't really want to put up the URL here, so sorry for being vague.

    Jar said: Mind pasting a sample of the logs? Easier to see patterns I might recognize.

    Here's the first/last few of each session on one particular day:

    217.12.215.96 - - [13/Jun/2014:11:51:22 +0000] "GET /content/439453 HTTP/1.1" 301 185 "http://some-website.com/content/439453" "Mozilla/5.0 (Windows NT 5.1; rv:5.0.1) Gecko/20100101 Firefox/5.0.1"
    217.12.215.96 - - [13/Jun/2014:11:51:22 +0000] "GET /content/439453 HTTP/1.1" 301 185 "http://some-website.com/content/439453" "Mozilla/5.0 (Windows NT 5.1; rv:5.0.1) Gecko/20100101 Firefox/5.0.1"
    217.12.215.96 - - [13/Jun/2014:11:51:23 +0000] "GET /content/439453 HTTP/1.1" 301 185 "http://some-website.com/content/439453" "Mozilla/5.0 (Windows NT 5.1; rv:5.0.1) Gecko/20100101 Firefox/5.0.1"
    217.12.215.96 - - [13/Jun/2014:11:51:23 +0000] "GET /content/439453 HTTP/1.1" 301 185 "http://some-website.com/content/439453" "Mozilla/5.0 (Windows NT 5.1; rv:5.0.1) Gecko/20100101 Firefox/5.0.1"
    217.12.215.96 - - [13/Jun/2014:11:51:23 +0000] "GET /content/439453 HTTP/1.1" 301 185 "http://some-website.com/content/439453" "Mozilla/5.0 (Windows NT 5.1; rv:5.0.1) Gecko/20100101 Firefox/5.0.1"
    217.12.215.96 - - [13/Jun/2014:11:51:23 +0000] "GET /content/439453 HTTP/1.1" 301 185 "http://some-website.com/content/439453" "Mozilla/5.0 (Windows NT 5.1; rv:5.0.1) Gecko/20100101 Firefox/5.0.1"
    217.12.215.96 - - [13/Jun/2014:11:51:23 +0000] "GET /content/439453 HTTP/1.1" 301 185 "http://some-website.com/content/439453" "Mozilla/5.0 (Windows NT 5.1; rv:5.0.1) Gecko/20100101 Firefox/5.0.1"
    217.12.215.96 - - [13/Jun/2014:11:51:23 +0000] "GET /content/439453 HTTP/1.1" 301 185 "http://some-website.com/content/439453" "Mozilla/5.0 (Windows NT 5.1; rv:5.0.1) Gecko/20100101 Firefox/5.0.1"
    
    ...
    
    217.12.215.96 - - [13/Jun/2014:12:21:59 +0000] "GET /content/439453 HTTP/1.1" 301 185 "http://some-website.com/content/439453" "Mozilla/5.0 (Windows NT 5.1; rv:5.0.1) Gecko/20100101 Firefox/5.0.1"
    217.12.215.96 - - [13/Jun/2014:12:21:59 +0000] "GET /content/439453 HTTP/1.1" 301 185 "http://some-website.com/content/439453" "Mozilla/5.0 (Windows NT 5.1; rv:5.0.1) Gecko/20100101 Firefox/5.0.1"
    217.12.215.96 - - [13/Jun/2014:12:21:59 +0000] "GET /content/439453 HTTP/1.1" 301 185 "http://some-website.com/content/439453" "Mozilla/5.0 (Windows NT 5.1; rv:5.0.1) Gecko/20100101 Firefox/5.0.1"
    217.12.215.96 - - [13/Jun/2014:12:21:59 +0000] "GET /content/439453 HTTP/1.1" 301 185 "http://some-website.com/content/439453" "Mozilla/5.0 (Windows NT 5.1; rv:5.0.1) Gecko/20100101 Firefox/5.0.1"
    217.12.215.96 - - [13/Jun/2014:12:21:59 +0000] "GET /content/439453 HTTP/1.1" 301 185 "http://some-website.com/content/439453" "Mozilla/5.0 (Windows NT 5.1; rv:5.0.1) Gecko/20100101 Firefox/5.0.1"
    217.12.215.96 - - [13/Jun/2014:12:21:59 +0000] "GET /content/439453 HTTP/1.1" 301 185 "http://some-website.com/content/439453" "Mozilla/5.0 (Windows NT 5.1; rv:5.0.1) Gecko/20100101 Firefox/5.0.1"
    

    [second session]

    5.34.179.75 - - [13/Jun/2014:15:49:13 +0000] "GET /content/535205 HTTP/1.1" 301 185 "http://some-website.com/content/535205" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:14.0) Gecko/20100101 Firefox/14.0.1"
    5.34.179.75 - - [13/Jun/2014:15:49:13 +0000] "GET /content/535205 HTTP/1.1" 301 185 "http://some-website.com/content/535205" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:14.0) Gecko/20100101 Firefox/14.0.1"
    5.34.179.75 - - [13/Jun/2014:15:49:13 +0000] "GET /content/535205 HTTP/1.1" 301 185 "http://some-website.com/content/535205" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:14.0) Gecko/20100101 Firefox/14.0.1"
    5.34.179.75 - - [13/Jun/2014:15:49:13 +0000] "GET /content/535205 HTTP/1.1" 301 185 "http://some-website.com/content/535205" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:14.0) Gecko/20100101 Firefox/14.0.1"
    5.34.179.75 - - [13/Jun/2014:15:49:13 +0000] "GET /content/535205 HTTP/1.1" 301 185 "http://some-website.com/content/535205" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:14.0) Gecko/20100101 Firefox/14.0.1"
    
    ...
    
    5.34.179.75 - - [13/Jun/2014:16:19:41 +0000] "GET /content/535205 HTTP/1.1" 301 185 "http://some-website.com/content/535205" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:14.0) Gecko/20100101 Firefox/14.0.1"
    5.34.179.75 - - [13/Jun/2014:16:19:41 +0000] "GET /content/535205 HTTP/1.1" 301 185 "http://some-website.com/content/535205" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:14.0) Gecko/20100101 Firefox/14.0.1"
    5.34.179.75 - - [13/Jun/2014:16:19:41 +0000] "GET /content/535205 HTTP/1.1" 301 185 "http://some-website.com/content/535205" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:14.0) Gecko/20100101 Firefox/14.0.1"
    5.34.179.75 - - [13/Jun/2014:16:19:41 +0000] "GET /content/535205 HTTP/1.1" 301 185 "http://some-website.com/content/535205" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:14.0) Gecko/20100101 Firefox/14.0.1"
    5.34.179.75 - - [13/Jun/2014:16:19:42 +0000] "GET /content/535205 HTTP/1.1" 301 185 "http://some-website.com/content/535205" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:14.0) Gecko/20100101 Firefox/14.0.1"
    5.34.179.75 - - [13/Jun/2014:16:19:42 +0000] "GET /content/535205 HTTP/1.1" 301 185 "http://some-website.com/content/535205" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:14.0) Gecko/20100101 Firefox/14.0.1"
    

    Here's another example

    5.34.179.65 - - [15/Jun/2014:04:51:28 +0000] "GET /listing/5478 HTTP/1.1" 301 185 "http://some-website.com/listing/5478" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
    5.34.179.65 - - [15/Jun/2014:04:51:29 +0000] "GET /listing/5478 HTTP/1.1" 301 185 "http://some-website.com/listing/5478" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
    5.34.179.65 - - [15/Jun/2014:04:51:29 +0000] "GET /listing/5478 HTTP/1.1" 301 185 "http://some-website.com/listing/5478" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
    5.34.179.65 - - [15/Jun/2014:04:51:29 +0000] "GET /listing/5478 HTTP/1.1" 301 185 "http://some-website.com/listing/5478" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
    5.34.179.65 - - [15/Jun/2014:04:51:29 +0000] "GET /listing/5478 HTTP/1.1" 301 185 "http://some-website.com/listing/5478" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
    5.34.179.65 - - [15/Jun/2014:04:51:29 +0000] "GET /listing/5478 HTTP/1.1" 301 185 "http://some-website.com/listing/5478" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
    
    ...
    
    5.34.179.65 - - [15/Jun/2014:05:23:47 +0000] "GET /listing/5478 HTTP/1.1" 301 185 "http://some-website.com/listing/5478" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
    5.34.179.65 - - [15/Jun/2014:05:23:47 +0000] "GET /listing/5478 HTTP/1.1" 301 185 "http://some-website.com/listing/5478" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
    5.34.179.65 - - [15/Jun/2014:05:23:47 +0000] "GET /listing/5478 HTTP/1.1" 301 185 "http://some-website.com/listing/5478" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
    5.34.179.65 - - [15/Jun/2014:05:23:47 +0000] "GET /listing/5478 HTTP/1.1" 301 185 "http://some-website.com/listing/5478" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
    5.34.179.65 - - [15/Jun/2014:05:23:47 +0000] "GET /listing/5478 HTTP/1.1" 301 185 "http://some-website.com/listing/5478" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
    5.34.179.65 - - [15/Jun/2014:05:23:47 +0000] "GET /listing/5478 HTTP/1.1" 301 185 "http://some-website.com/listing/5478" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
    

    All responses are 301 because this is a HTTPS only site and they're only making HTTP requests (so nginx sends a 301 redirect, which the bot isn't following).

  • geekalotgeekalot Member
    edited June 2014

    ...

  • xyzxyz Member

    Appreciate the suggestion geekalot, though I'm more interested if anyone has theories on the 'why', as opposed to how to deal with them :)

Sign In or Register to comment.