All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Weird DoS attempt
For the past few days, I've been getting a flood of HTTP requests at times of the day. However I don't understand the odd and consistent pattern of these attacks - here some facts:
- There's exactly 2 attack sessions every day
- Each session lasts for about 30 minutes (not exact, but usually quite close); I haven't yet seen the sessions overlap and the sessions appear to be 1-4 hours apart (ie one session will finish, then the next starts a few hours later)
- Only a single URL is hit during each session
- Each session comes from a new IP (have seen IPs reused); the user agent also gets randomized (probably picked from a list of common UAs) and a new URL is selected. Selected URLs seem to be random as I can't see any pattern
- IPs are all from Ukraine and a number of them seem to be from the same network
- The rate is around 23 reqs/sec and doesn't seem to have any adverse affects on the site, since the webserver seems to handle it fine
If the above was a bit confusing, consider this as an example:
on 2014-06-15 at 10:00, IP 1.2.3.4 starts continuously requesting /content/33, then stops at 10:30
and on the same day at 13:20, IP 5.7.8.9 starts continuously requesting /listing/8 until 13:51
Each session results in around 42,000 requests being made.
This behavior seems odd to me and I don't understand the reasoning behind it. Would anyone here have any clue?
If a DoS attack is meant to try to take down a website, why would the attack stop after half an hour? Why would there be only two attempts every day? Also, if the attack doesn't seem to be working, why would the attacker keep trying the same strategy over and over again, without making any modification?
Comments
Slowloris?
RUDY?
Seems like one of those really stupid skiddy booters, but it might be a scaper or something else that's simply intent on indexing a single page at a time [ really wouldn't surprise me, we've seen odd traffic patterns before, but not quite like the one you described. ]
How many requests in each 30 minute period ?
it seems like bad bots to me.is that site popular?
Around 42,000 (so average of 23 requests/second). I don't think a scraper needs to make that many requests to a single URL...
RUDY?
Thanks for the pointers there. Doesn't appear to be though - the requests are all GET. I can't tell if the requests are holding connections open for a long time, but nginx here is set to 1500 max connections, so, if I understand it correctly, it would've been exhausted way before 42k requests hit?
Stuff like that will kill apache in seconds, anything nginx/lighttpd will have no issue with that type traffic on static files.
Quick and dirty solution could be http://deflate.medialayer.com/
Thanks for the tip @linuxthefish
Using nginx here - I probably should probably make use of the limit_conn module, although concurrent connection limits don't appear to be the issue, assuming nginx logs those.
Been getting something similar on one of my sites, top country to visit is Ukraine.
Its usually from random IPs as well, site has nothing to do with anything outside of the US. It does have a phpbb forum that they seem to try and register on constantly but the anti-bot thing seems to stop that.
What is on said URL?
Mind pasting a sample of the logs? Easier to see patterns I might recognize.
Nothing particularly outstanding - URLs that a typical user might browse. It's not the index/home page, and links to these pages are often buried several pages deep (ie you'd have to navigate through a few pages before you found the ones they're targeting). As the IP never requests anything other than this page, it's either found it via a search engine or someone put in a list of URLs to target (latter seems unlikely as the URLs don't seem to have much similarity, for example, one might be an article from 2012, another might be from more recent, another might be an old listing page etc).
I don't really want to put up the URL here, so sorry for being vague.
Here's the first/last few of each session on one particular day:
[second session]
Here's another example
All responses are 301 because this is a HTTPS only site and they're only making HTTP requests (so nginx sends a 301 redirect, which the bot isn't following).
...
Appreciate the suggestion geekalot, though I'm more interested if anyone has theories on the 'why', as opposed to how to deal with them