New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Setting up advanced HTTP Flood protection
PrincessOfCats
Member
in Help
After multiple massive HTTP floods on a friend's site I was hosting (I already had protection for other attacks), I have been working on a cookie/js verification based on what CF has. As much as I would like to use CF, its let me down a few too many times in the past.
So far, I understand what I am doing with the cooke/js verification. The main issue is that the page needs to be displayed before anything on the site is displayed.
So far, I have only had one idea, and that is requiring a valid cookie to access any page other than the default page where I do the cookie/js verification. Is there a better way to do this?
Comments
edit: nvm, figured it out.
There is actually a nice nginx module to do this: https://github.com/kyprizel/testcookie-nginx-module
I currently use a haproxy -> varnish setup, but I can slip nginx in between to do the verification.
What Ive decided:
Haproxy currently handles the rate limiting swimmingly. When it gets to a certain number of requests/time, it swaps the ip to point at a different backend (in this case, nginx). Nginx will do the testcookie to see if stuff is allright, if it is, it will allow the request. If not, it will display an error page.
You could take a look at nginx testcookie it might work for you
Interesting setup, but have a few questions if you don't mind:
Varnish has no https. I use HaProxy as a SSL terminator as well as a rate limiter. Nginx's rate limiter is not as good as I hoped and it fell flat on its face a few times (the requests just started timing out even though max connections were above what was being done)
Iptables causes ips using NAT (Universities/etc especially, who decide to use a single IP for all outbound student communcations. Some use multiple for wifi/ethernet or whatever) to trip and fall flat on their face without any reason. I used to do rate limiting only with haproxy. This was one of the reasons that I wanted to do cookie/js verification. People who share IPs can still access the site without getting rate limited.
With this method, I can display a page for who to contact to tune some stuff in case they get banned accidentally. I highly dislike the fact that people have no idea where to go to get unbanned because they are banned.
For the cookies part, read the description, the page sets the cookie, and if there is no valid cookie, nginx goes back to the cookie-setting page for a limited number of times.
Side note: Ideas about setup were borrowed from Reddit rate limiting and CF.
Simple yet ingenious Thanks!
Also, if some people are DDOSing you from a school (This was actually the cause of me needing to rate limit...) and the IT Department doesn't give a crap, you won't end up blocking everyone in the school, just those that don't pass the JS/Cookie test.
This method can be bypassed by specialized 'bots' that some booters have these days (they actually do the JS/Cookie stuff!), but I plan on further working on it to get some more protection when time comes.
Chances are, by the time those become extremely popular, my DDOS protection will have to be upped before I start tinkering with the HTTP flood protection
If you want to test on Ubuntu (sorry centos users....), see https://launchpad.net/~phoenix-o/+archive/ubuntu/nginx-security/+packages
It will require apt pinning due to debian revisions.
And maybe you can use fail2ban to monitor nginx logs. For http flood it might work.