New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Anycast routes latency.
Why sometimes anycast paths have terrible delays to certain providers/datacenters?
Is this has to do something with a lack of peering?
On traceroute, it seems like packets literally running circles around.
151 ms to L.A., 122ms to Dallas, 94ms to VA...
Canada, BC, Vancouver Telus 0% 33 5.59 6.06 5.46 6.68 0.51 Canada, BC, Vancouver Shaw 0% 33 16.45 15.31 13.12 18.36 1.45 USA, CA, Fremont Hurricane FMT2 0% 33 9.65 9.59 9.53 9.72 0.04 USA, CA, Fremont Linode 0% 33 25.66 25.82 25.24 26.76 0.4 USA, CA, San Francisco Digital Ocean 0% 33 22.27 22.89 22.22 23.33 0.41 USA, CA, Los Angeles QuadraNET 0% 33 151.5 151.65 151.11 152.59 0.33 USA, CA, Los Angeles Vultr 0% 34 1.15 1.26 1.1 4.79 0.62 USA, CA, Seattle Google 0% 33 25 25.03 24.43 25.47 0.21 USA, CO, Denver Cogent 0% 32 23.77 23.68 22.78 24.23 0.43 USA, TX, Dallas Softlayer 0% 33 122.2 123.39 121.83 129.3 2.13 USA, IL, Chicago Cogent 0% 32 13.89 15.83 13.79 25.04 2.59 USA, GA, Atlanta 55 Marietta/RamNode0% 33 15.24 15.04 14.61 16.06 0.25 USA, VA, Vint Hill OVH 6.3% 32 94.09 92.61 90.73 94.9 1.47 USA, NY, New York Telehouse/RamNode 0% 32 17.24 17.36 17.09 18.17 0.31 Canada, QC, Montreal OVH 0% 32 96.04 94.73 93.57 96.15 0.94
This one is especially terrible with across the pond jumps.
(an .icu TLD nameserver actually)
Canada, BC, Vancouver Telus 0% 15 143.27 143.63 143.22 144.32 0.32 show Canada, BC, Vancouver Shaw 0% 14 158.71 157.56 155.39 159.95 1.75 show USA, CA, Fremont Hurricane FMT2 0% 15 59.57 59.79 59.41 60.65 0.44 show USA, CA, Fremont Linode 0% 15 192.55 192.95 192.52 193.58 0.38 show USA, CA, San Francisco Digital Ocean 0% 15 193.94 193.59 193.11 194.11 0.32 show USA, CA, Los Angeles QuadraNET 0% 15 57.38 57.47 57.1 58.07 0.27 show USA, CA, Los Angeles Vultr 0% 15 187.48 187.28 186.63 188.58 0.44 show USA, CA, Seattle Google 0% 15 57.49 57.48 56.95 57.84 0.25 show USA, CO, Denver Cogent 0% 14 26.91 26.31 24.84 26.96 0.82 show USA, TX, Dallas Softlayer 0% 15 121.22 121.79 121.14 123.44 0.58 show USA, IL, Chicago Cogent 0% 14 2.86 3.54 1.99 7.61 1.69 show USA, GA, Atlanta 55 Marietta/RamNode 0% 15 18.07 18.63 18.07 19.14 0.39 show USA, VA, Vint Hill OVH 0% 14 208.36 208.01 207.39 208.62 0.34 show USA, NY, New York Telehouse/RamNode 0% 14 2.17 2.15 1.7 2.2 0.13 show Canada, QC, Montreal OVH 0% 14 203.62 204.16 203.62 205.32 0.4 show
Comments
iirc Anycast will route you through the shortest route (least hop) unless manually overriden, and shortest route doesn't always mean smallest ping
Shortest ASN path. But it boils down to the same thing. Really hard to manage if you have inconsistent set of upstreams
BuyVM's homemade anycast looks quite nice overall:
All within 45ms in US and EU, except Shaw
Francisco, please please add Singapore!
If the staff too tired with manual overrides we see jumps across the ocean.
Wonder if Cloudflare has a whole department fighting anycast troubles constantly...
You don't have to do nearly as many manual tweaks when you're on almost every major IX in the world.
Not a fan of their free speech laws.
Tokyo?
Francisco
Tokyo is just 120ms away from LasVegas. If Singapore is not an option then maybe Malaysia or Sydney.
Malaysia has Thailand,Vietnam,Indonesia,India and even Perth within 50-60ms reach. The population is crazy huge out there.
Sydney has all Australia covered plus New Zealand, Fiji and a whole bunch of Pacific islands with exception of French Polynesia which is connected via Hawaii.
Fairly sure Malaysia is in a similar spot for speech.
AU is interesting, would have to test the latency to see if it would be good.
I got a good offer in Singapore but I think it would be a bit hypocritical to go on about freedom of speech then host in a country without it.
Francisco
Optimizing Anycast routes is challenging and time consuming. Not everyone does a good job. Not every route is 'fixable' although 90%+ are. It gets even more difficult when the other networks are heavily (manually) tuned or Anycast themselves.
@umi if you are a customer have you thought to point out bad routes to the provider of the network? Sometimes people are just unaware of a route / configuration change (particularly in the low end space).
I'm betting they don't have to deal with as much bullshit from ISPs that refuse to peer as well.
Yes, please consider AU
+1
A server in Singapore can serve> @SplitIce said:
It looks like a terrible issue with anycast providers and a business model for some.
They are well aware of it, and continue on selling xx anycast PoPs...
It's easier to name a few providers that have done ipv4 anycast right:
I use some .club and .co domains. Their TLD namservers' anycast ips are from ULTRADNS - NeuStar, Inc. ASN and have no big routing issues worldwide. I can recommend them.
Cloudflare anycast nameservers that I managed to test are fast and have no big routing issues except once a year July downtime Best of free dns options.
I'm testing BuyVM's anycast and I'm gonna use it in production soon - it is good and uniform, but the South-EastAsia/Australia regions have a blind spot with 160-180ms RTT to nearest server. For just a 3 of them it is a masterpiece I think.
The geek-solutions.ca came here asking for test of the new anycast product, I signed in, but still nothing heard from them for many weeks.
The .icu .xyz anycast nameservers are operated by CENTRALNIC-ANYCAST-E, CENTRALNIC-ANYCAST-F ASNs and are very bad at routing.
CLOUDNSNET has some issues in US on their Premium S tier anycast nameservers.
VULTR anycast nameservers have some routing issues worldwide, but overall, it is a decent free dns service.
HE.net can do AXFR but the only decent anycast nameserver is ns4.he.net. Others have routing issues in NewYork and other places. Not actually that bad for a free service. ns4.he.net can be mixed with your own nameservers.
@umi you can always use our Anycast.
Please do it, sir
I got the same reasoning as you - SG's freedom of speech stance is really concerning.
Tokyo on the other hand is much better - very few competitions there (Vultr/Linode), decent latency to US. My head just aches when I see traceroutes from SG doing SG -> US -> EU resulting in 300ms+ latency.
And your favourite transit providers (Cogent + HE) are both present in JP although neither of them has a strong Asian presence.
Thanks boss, I appreciate it.
Honestly I'd probably prefer to buy into someone elses network for an Asia roll out.
That, or just whore myself to NTT.
Francisco
Yeah, that's easier probably easier, except you will now have tickets after tickets begging you to fix their anycast routing since the bandwidth mix are different among the 4 locations
Very true!
Who knows what will happen, but we need to upgrade LU to Ryzen before we move into new digs
Francisco
If the anycast routing stability will be better in Tokyo then sure go for it!
For what its worth our anycast will be handled by Cloudflare sometime this year. They intend for it to be purely latency driven, basically argo optimized anycast ips.
Neat right?
Francisco
Asia location would sure be nice. :P Some alternative for OVH Anti-DDoS.
@stefeman we are in Asia (Singpaore and Tokyo) and are regularly used expecially in Singapore a protection for peoples OVH servers
Do you offer AnyCast without DDoS mitigation? The bandwidth requirement for me with AnyCast is outside the budget of what I would need but I have looked X4B many times on and off.
@daxterfellowes
No we don't. We used to do non-protected services 5+ years ago... and they constantly got attacked. I sleep alot better now.
Could you expand further on your experiences with CloudDNSnet anycast/GeoDNS by chance?
Responce time >90ms within US for a nameserver is a bit slow. But with some not so small providers it is so:
Here is the check of their nameservers (from control panel) with ping.pe:
all 4 nameservers show same slowdown in L.A. , Dallas, VintHill
Here is the Cloudflare's nameserver to compare:
Just a little glitch with HE in Freemont.