Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Google public dns issues (SERVFAIL), build own resolver?
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Google public dns issues (SERVFAIL), build own resolver?

dccdcc Member, Host Rep

Looks like every ~20th DNS request to 8.8.8.8 fails with SERVFAIL. This is going on for at least 16 hours now, confirmed from multiple locations.

I wonder if anyone else noticed this?

Any best practices on building own resolver?

«1

Comments

  • You don't need to build your own resolver. If it'll be internet accessible then you'll be stuck dealing with the security issues around open resolvers.

    Diversity your DNS servers.

    I personally use 8.8.4.4 (Google) and 74.82.42.42 (HurricaneElectric), but there are tons of other resolvers that aren't google that you can use.

    Thanked by 1dcc
  • The ISP I work for had Google so kindly start rate limiting our IP ranges, because we use their DNS.

    That was...fun. We now run/maintain our own dns server.

  • vdedivdedi Member

    howtoforge.com or digitalocean have alot tutorials to do this, you can use unbound, maradns or any dns servers, or use public dns.

  • OpenDNS all the way, switched from google dns because of several issues a few years back.

    Thanked by 2netomx klikli
  • dccdcc Member, Host Rep

    Appreciate the suggestions. Added HE resolver (74.82.42.42) into the existing pool of Google and Level3 resolvers and tweaked timeouts, everything is better now.

    Regarding opendns, didn't opendns return an IP pointing to a page loaded with ads instead of returning NXDOMAIN? I did a quick check, does not seem to be the case now?

  • KrisKris Member

    No, it's not the case now.

    Feel free to add 208.67.220.220 and 208.67.222.222 to the resolvers and use options like rotate to diversify the lookups.

    The 4.2.2.x series to screw with NX domains now. They were good pre-Google, but in the last few years they screw with NX, so I rarely use them.

  • HackedServerHackedServer Member
    edited April 2016

    @dcc said:
    Regarding opendns, didn't opendns return an IP pointing to a page loaded with ads instead of returning NXDOMAIN? I did a quick check, does not seem to be the case now?

    Level3 resolvers do that.

    ;; ANSWER SECTION:
    notarealdomain.nxdomain. 10 IN A 198.105.244.11
    notarealdomain.nxdomain. 10 IN A 104.239.213.7
    ;; Query time: 39 msec
    ;; SERVER: 4.2.2.2#53(4.2.2.2)

    Sounds like its a great opportunity for you to run namebench and find the best DNS servers for you.

    https://code.google.com/archive/p/namebench/

    Thanked by 1dcc
  • dccdcc Member, Host Rep

    Looks like there have been some major changes recently, L3 hijacking is news to me. Already messing around with namebench, I had no idea there are so many open dns resolvers out there.

  • @dcc said:
    Regarding opendns, didn't opendns return an IP pointing to a page loaded with ads instead of returning NXDOMAIN? I did a quick check, does not seem to be the case now?

    No longer the case, not sure this happened before or after Cisco acquisitions though. One thing to keep in mind that opendns is advertised as a home solution, I know they have a VIP version for $20 a year, not sure about business versions.

    Thanked by 1dcc
  • dccdcc Member, Host Rep

    @TheOnlyDK thanks! I should have mentioned in my OP that we are definitely a "heavy user". I guess we will add it to the pool and see how it goes, I suppose the worst that could happen is we get banned.

  • @dcc said:
    TheOnlyDK thanks! I should have mentioned in my OP that we are definitely a "heavy user". I guess we will add it to the pool and see how it goes, I suppose the worst that could happen is we get banned.

    I don't see why you would get banned for using a public DNS. They didn't apecifilly say it is for home users only, just the name is called OpenDNS Home (and OpenDNS Home VIP or something for the VIP version). I use OpenDNS for all my boxes, didn't have a single issue, though I'm not that heavy of a user, so don't quote me on that.

    Thanked by 2dcc netomx
  • Location matters of course. If you're in Europe this is a very good one:

    anycast.censurfridns.dk / 91.239.100.100 / 2001:67c:28a4::
    ns1.censurfridns.dk / 89.233.43.71 / 2002:d596:2a92:1:71:53:: 
    

    Also, don't forget OpenNIC servers. Their site automatically shows the ones nearest to you.
    https://www.opennicproject.org/

  • NyrNyr Community Contributor, Veteran

    There aren't many good public recursors. I had been tracking this situation over the years so I think I can be of help :)

    • Level 3 did hijack globally, then they stopped, then they did based on your geographic location, not sure if it's still the case. I don't think they can be trusted any longer.
    • Google had some small issues over the years which made them not ideal.
    • HE has been reliable over the years and they got plenty of locations for their anycast pool. A good choice.
    • NTT has also been good over the years. Nothing I can complain about.
    • OpenDNS is not suitable for very heavy users. They also have "malware/phising protection" by default, including false positives. They are only good for personal use if you register and disable their filters for your IP.
    • Yandex is unreliable. Also, some US domains can't be resolved from RU IP addresses.

    So my current suggestion is HE + NTT which has been working wonderfully for a long time.

    If anyone knows about any other NS which:

    • Has anycast
    • Doesn't hijack
    • Is stable and reliable

    Please let me know (but I don't think there are many more, except some regional ones in China for example).

  • Ole_JuulOle_Juul Member
    edited April 2016

    Nyr said: If anyone knows about any other NS which

    https://blog.uncensoreddns.org/ (as mentioned above) doesn't fit the bill?

  • rm_rm_ IPv6 Advocate, Veteran
    edited April 2016

    Keep in mind that if you put more than 3 servers into /etc/resolv.conf, only the first 3 are actually used.

    Thanked by 2vimalware Rolter
  • edited April 2016

    Some others to consider:

    DYN:

    216.146.35.35 (Dyn 1)
    216.146.36.36 (Dyn 2)

    Ultra:

    156.154.70.1 (Ultra 1)
    156.154.71.1 (Ultra 2)

    OpenNIC:

    128.173.89.246
    50.116.38.157
    205.185.120.143
    74.207.247.4

    Thanked by 1vimalware
  • NyrNyr Community Contributor, Veteran
    edited April 2016

    Ole_Juul said: https://blog.uncensoreddns.org/ (as mentioned above) doesn't fit the bill?

    While I appreciate his effort, the anycast server doesn't PoPs outside of DK.

    Very interesting, looks like this was launched recently and they provide an unfiltered NS.

  • @HackedServer said:
    You don't need to build your own resolver. If it'll be internet accessible then you'll be stuck dealing with the security issues around open resolvers.

    Not really. All you need is a whitelist and your pretty much done.
    Using your own DNS will about always be fastest and best.

  • NyrNyr Community Contributor, Veteran

    TehEnforce said: Using your own DNS will about always be fastest and best.

    I guess you say this because either:

    • You haven't ever set up a recursor
    • You didn't measure how fast the resolution was

    Since public recursors have most records already cached, they will usually be faster. Running your own recursor makes waiting more than a full second to retrieve a record not uncommon.

    Thanked by 2vimalware Dylan
  • info_hashinfo_hash Member
    edited April 2016

    Nyr said: Running your own recursor makes waiting more than a full second to retrieve a record not uncommon.

    I tried unbound and pdnsd and rarely did experience such long wait.

    3/400ms happens (or even more than 1 sec as you say) when it needs to fetch the information on the other side of the world but requests are generally faster. And once the result is cached, well, that's as fast as RAM, and it's pretty secure and there is no blocking :)

  • @Nyr said:
    Since public recursors have most records already cached, they will usually be faster. Running your own recursor makes waiting more than a full second to retrieve a record not uncommon.

    I have been running and managing my own BIND recursor for over 2 years so its safe to say I know what I am talking about.

    And yes I compared it with my local ISP's and other DNS services

    Like @info_hash said.

    That claim is not true at all. Yes first startup will take longer but thats everywhere. Windows, android so yes even DNS but than answers will be about instant.

  • ricardoricardo Member
    edited April 2016

    Nyr said: There aren't many good public recursors. I had been tracking this situation over the years so I think I can be of help :)

    http://www.tummy.com/articles/famous-dns-server/ ?

    -

    I run unbound (only accessible by 127.0.0.1) & a custom C prog behind them on many VPS' I have. The program just does some async DNS lookups of domains, privately.

    I got accused of it being A DDOS program by Quadranet/Crissic today, though I suspect I just triggered some flagging software & level 0.1 support said it was DDOS.

    Apparently VPS can be too low end to do some simple DNS stuff :)

    I'd recommend running something that caches lookups, whatever you use further upstream. FWIW I do have the nameservers of pretty much every domain in existence... doing a count of each unique one would likely lead to a fat list of free public DNS.... though it seems most are listed already.

  • NyrNyr Community Contributor, Veteran

    TehEnforce said: I have been running and managing my own BIND recursor for over 2 years so its safe to say I know what I am talking about.

    And yes I compared it with my local ISP's and other DNS services

    Then you would've found that you could likely reach your ISP in 1-50 ms while your recursor would struggle a lot to resolve an uncached record in the same time.

    You said that using your own DNS would always be faster and that's certainly not true. For a low volume of queries, it's actually the opposite.

    I did read that some years ago. Sadly, since they started hijacking, the service is not interesting to me anymore.

    ricardo said: I got accused of it being A DDOS program by Quadranet/Crissic today, though I suspect I just triggered some flagging software & level 0.1 support said it was DDOS.

    Are you sure the NS was secure/private? If so explain that to them and they shouldn't have a problem...

  • Nyr said: Are you sure the NS was secure/private? If so explain that to them and they shouldn't have a problem...

    Yes, it binds to 127.0.0.1 and I'm the only user on the server. They did back off when I said my "malicious script" was simply doing DNS lookups, I offered them the source but was just given boilerplate vague responses. unbound obviously helps in politeness as its my nameserver the program uses, and caches anything worth caching.

    I should clarify in the case of this thread, its purpose is to lookup thousands of domains and is threaded, but it's not CPU heavy and at maximum is doing around 10 a second. It's deliberately rate-limited to 5 threads and only runs a few days of the month. Its maximum load is around 0.01. To me that's quite low-end, but it's something to be aware of on these low end hosts...

  • @Nyr said:
    Are you sure the NS was secure/private? If so explain that to them and they shouldn't have a problem...

    Again. You can't compare cached vs uncached response times when you compare DNS A vs DNS B.

    My DNS never struggled to get uncached records and I never waited for more than a few 100 ms before my DNS found and cached the records.

    And once its cached its even faster than my ISP and other DNS services.

    I suggest you try to run your own so you can see that I am right.

  • NyrNyr Community Contributor, Veteran

    TehEnforce said: You can't compare cached vs uncached response times when you compare DNS A vs DNS B.

    I agree to an extent, but you need to understand that if 99 % of the records are already cached, it will obviously be faster. We are just talking about what is faster.

    TehEnforce said: My DNS never struggled to get uncached records and I never waited for more than a few 100 ms before my DNS found and cached the records.

    500+ ms are very common, with 1 second not being very rare depending on the domain NS, TLD, etc...

    TehEnforce said: And once its cached its even faster than my ISP and other DNS services.

    Yeah, obviously.

  • Been having issues with Google DNS as well. They've either rate limited some of our more popular servers or they're just having issues. It's been happening the past 72 hours.

    Thanked by 1dcc
  • KrisKris Member
    nameserver 208.67.220.220
    nameserver 8.8.8.8
    nameserver 208.67.222.222
    nameserver 8.8.4.4
    
    options rotate 
    options timeout:1
    options attempts:3
    

    ymmv

    Thanked by 2geekalot netomx
  • shovenoseshovenose Member, Host Rep

    I switched to opendns yesterday. Internet feels snappier now.

    Thanked by 1netomx
Sign In or Register to comment.