Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


GeoIP DNS Setups.
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

GeoIP DNS Setups.

ElliotJElliotJ Member
edited April 2012 in General

Hey guys,

I'm working on a CloudFlare-esque reverse proxy system. Loads of LEBs in different locations, that sort of thing.
Everything's going great, but I'm trying to work out the best way of attacking the DNS issue.
I've never set-up nameservers before, instead leaving them up to the people who 'know what they're doing'. That being said, I'm willing to set them up this time round.

So, the question is:
Do you know any (hopefully free) DNS providers that play nice with GeoIP DNS?
If the answer to that is a resounding 'No.', what's the most elegant way of setting up the name servers? I'm guessing that they'll need to lookup the IP database, so query response time here will be key.

If anyone's had experience of this, or is up for a nerdy challenge, please chime in :>

Cheers!

Comments

  • joepie91joepie91 Member, Patron Provider

    I believe there's a very solid GeoDNS extension for PowerDNS, you might want to have a look at that.

  • Cheers :)
    I looked into it, but it turns out I've managed to get it working through GeoScaling.com

    You have to do a little tweaking to get it working, but it turns out great
    ... As long as you don't use OpenDNS.

  • You can use gdnsd for GeoIP DNS

  • AkiraAkira Member

    @ElliotJ Seems interesting, can you write more about what you are up to?

  • PowerDNS + Mysql replication = <3

    Thanked by 1Francisco
  • I'm working out a way to create a low cost, scalable Magento platform. Although it's a solid e-commerce app, it can be pretty resource intensive.
    So, I'm throwing in loads of geographically diverse reverse proxies running nginx to cache static files and then I'm also trying to find a way to mirror the 'main server' where the sites are held - Need to find a suitable way of scaling that.
    Redundancy + Performance under... $20/month? Should be doable.

    GeoScaling is working out great, not only is it providing GeoIP DNS, but it also has server monitoring, so it can divert traffic in-case of high load on a particular server without any intervention.
    The load monitoring is working great for the reverse proxies at the moment, but it would be great if that could work on the back-end as well.

    I'm writing everything up as I go, intending on releasing some information with all the 'gotchas' I've encountered along the way.

    tl;dr version: geeky, geographically aware, load balancing over DNS, e-commerce platform, on a budget.

  • OliverOliver Member, Host Rep
    edited April 2012

    @ElliotJ I've got some experience with Magento and I don't honestly think what you are doing will make a huge difference. At the end of the day Magento sucks because it is so database heavy. As we all know from our LEB setups any current system can serve a lot of static traffic without any problems. It's database heavy stuff that kills performance.

    Basically what I am saying is that even if you hosted 99% of static content on a separate server, since serving the static files uses so few resources in the first place it won't make a huge difference. It's better to spend time optimising Magento (e.g. regular database cleaning, disabling a lot of logging features, and other tweaks) IMO.

  • @Oliver
    Absolutely, the database hog needs to be sorted out, so I'm getting used to the nginx cache settings, specifically the rules you can apply to specific directories and file extensions.
    I'm trying to cache the majority of the PHP responses, without jeopardising security. Hopefully that'll reduce the database queries, reducing most of the load.

    The main issue is over-caching, whereby literally anyone could get into the admin area without authentication, because it's been cached.
    Similarly, this could be an issue for the shopping cart and billing area; I'm working on excluding most of those critical files from the cache, but it's definitely a WIP.

    The idea is to allow automated set-up onto the platform with the pseudo-CDN and caching already built in. Theoretically, it should play nicely.
    I've tested Wordpress on it with PHP caching and managed to get a 300ms page load through Pingdom with the stock theme, no plugins, no optimisations.

  • OliverOliver Member, Host Rep

    Another thing that I am sure slows it down is the way it modifies the htaccess files for mod_rewrite (or whatever equivalent with non Apache setups) to create rewrite rules for each product category. I've got a customer who has hundreds of categories and Magento has gone and generated a huge htaccess file which I am sure is less than ideal from a performance perspective.

  • .htaccess is a pain, but you can reduce some of it if you force Apache to only look for .htaccess files in the web server root. Not sure if it'll work for Magento, but it's worth a try.
    I'm presuming Apache 2.4 will be able to handle them a little nicer as well... Well... I hope.

    The theory is primitive, but working at the moment.
    If one site is getting 100 requests/sec, presuming they're all not logged in, instead of re-generating that page for every user, you can just send over the cached version.
    Supposing you refresh the cache every hour, you're still effectively only needing to hit the backend for each page once an hour instead of, perhaps, a few hundred thousand times.
    A PHP cache extension could work for this, but it'd end up gobbling the limited ram resources. You could also force a plugin to generate a static version of the pages, but that'd also cause additional load.
    By diverting it onto other servers, you're getting the cached version; bypassing the database hit, the htaccess file, and the PHP generation.
    Not only that, but since the server the user will be connecting to will be geographically closer, they'll theoretically get a faster response.

    Once I've got that to work smoothly, the issue will be retrieving the files that absolutely can't be cached (billing area, admin area) without causing too much latency.
    I'm betting on the connection between the servers being superior to the user's local ISP, therefore making the pages load without too much hassle, whilst still retrieving the CSS/JS/Images from the cache.

    No idea if this will work out as planned; the security issue is my main worry >_>

  • OliverOliver Member, Host Rep

    Not 100% sure on this but I think Magento drops .htaccess files all over the place to prevent direct access to certain directories and a bunch of other stuff.

    Only other issue with caching is impatient clients who complain if they can't see their changes immediately. ;-)

  • True. Just set the 'expected support response time' to 1 hour and 1 minute, problem solved.

    Or, find a way to make nginx store the cache in directories based on the website hostname then include a 'Clear Cache' button somewhere that'll send a command to the cache servers to rm everything in the directory.
    That could get messy though.

  • OliverOliver Member, Host Rep

    I like the expect support response time, with a default response saying: "Please refresh the page to confirm the problem exists before contacting us".

    ;-)

Sign In or Register to comment.