Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


How VolumeDrive broke the Internet (part oft it)
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

How VolumeDrive broke the Internet (part oft it)

«1

Comments

  • Atrato really needs to get their shit together.

    Thanked by 2aus ihatetonyy
  • BrianHarrisonBrianHarrison Member, Patron Provider
    edited September 2014

    Did Atrato not have prefix filters or a max prefix limit in place? I'd say that's more appalling than VolumeDrive slipping up and announcing the global routing table.

    Leaking the routing table can happen. It's a relatively easy mistake to make -- it can be as simple as typo'ing the name of your outbound prefix filter list on a sizable peer group and all of a sudden you're blasting all of your peers with every route in your routing table. Just this evening AEBC leaked the global routing table to peers on the SIX for about an hour.

    But out of all your peers, you'd expect your transit provider to filter those leaked routes.

  • I wonder if this is the reason why they take 2weeks to setup a new dedi server :(

    Thanked by 2Jeffrey ihatetonyy
  • ExpertVMExpertVM Member, Host Rep

    We are hit with SIX route leak as well with one of our US server. Looks like such case is happening quite often.

  • Not even VD's fault: http://en.wikipedia.org/wiki/512k_day

    You guys (carriers) need to fix your shit, and stop using 10 year old Cisco router.

  • Why don't you, uhm, read the article before commenting?

    Thanked by 2GIANT_CRAB lbft
  • @Wintereise said:

    Last month it was Comcast, what's your point? This is going to keep happening until either Cisco releases software that isn't shit, or people stop using 10 year old routers in general. The 512k route limit is a huge problem that people have seen for a long time now, it's not as if this "suddenly happened".

    Verzon broke it a month earlier, this pretty much happens every month now.

  • dude, it was a routing leak.

  • trewqtrewq Administrator, Patron Provider

    @GoodHosting said:

    Did you read it? It had nothing to do with a route limit.

  • @trewq said:

    Right, so their router leaked extra routes, this still wouldn't have been an issue if the 512k limit wasn't a thing (as it has been every other month the internet has been affected.) This still does nothing to disprove the fact that this situation could have been avoided by proper planning on anyone's parts.

  • WintereiseWintereise Member
    edited September 2014

    GoodHosting said: Right, so their router leaked extra routes, this still wouldn't have been an issue if the 512k limit wasn't a thing

    No, sir. Why don't you, like, re-read it?

    Why would I want my traffic to be routed through Atrato due to their inability to properly set maxprefixes on their transit customers?

    It didn't increase the size of the FIB due to the leak, what it did it route vast amount of traffic not destined for Atrato through Atrato. This introduced subpar routing, as renesys so kindly pointed out.

    Please do not comment on things you have zero understanding of, and please then don't try to continue it after being called out.

    This is silly.

  • Interesting share. So, the internet can very much break ... :)

  • GoodHostingGoodHosting Member
    edited September 2014

    @Wintereise said:

    It's pretty silly you won't accept that the issue at hand is much bigger than Atrato being retarded, while the "sub-par" routing (which frankly, the entire United States of America is a shit route for the rest of the world), the 512k limit will continue to be a much bigger threat on the internet as a whole; and nobody is man up enough to deal with the issue, nor will the ball out the required money to buy a router from this decade.

    I find it sad you're ignoring the bigger picture, enjoy your little glass house.


    I mean shit, if anyone gave our friend @jnguyen the ability to broadcast and route, I bet he'd manage something on a lot larger scale than the opening post's explosion. As per the "who broke the internet", this will continue to be an issue; regardless of who starts each individual issue, the bigger issue is not going to just "go away" without people actually manning up or balling up.

  • GoodHosting said: I find it sad you're ignoring the bigger picture, enjoy your little glass house.

    Not quite.

    That is indeed an issue (which hopefully all service providers have already patched around), what they haven't patched around is incorrect prefixes being flooded in from providers who have essentially been given tier 1 telco status.

    Your issue and this are completely separate, so...

    Thanked by 1GIANT_CRAB
  • @Wintereise said:

    Alright then, fair enough. I figured you were blindly ignoring the bigger picture.

  • @GoodHosting said:
    I mean shit, if anyone gave our friend jnguyen the ability to broadcast and route, I bet he'd manage something on a lot larger scale than the opening post's explosion. As per the "who broke the internet", this will continue to be an issue; regardless of who starts each individual issue, the bigger issue is not going to just "go away" without people actually manning up or balling up.

    I'm just going to point this out...

    The issue you are expressing (512k limit) is irrelevant to this issue with Atrato. @Wintereise was simply stating that the "fix" for 512k limit wouldn't apply to this issue with Atrato. Then you got all defensive.

    There's no reason to be angry. lol.

    Thanked by 2mpkossen Infinity
  • @HalfEatenPie said:
    There's no reason to be angry. lol.

    This is LowEndTalk, the hard-coded default reaction is angry. That being said, the routes and traffic explosion wouldn't have hit so hard if the 512k issue wasn't there (as you can check the global routing around the same date, we broke 512k a few times the same day; although in different places.)

  • GIANT_CRABGIANT_CRAB Member
    edited September 2014

    sir sir Sir sir Sir

    Thanked by 3Wintereise trewq lbft
  • GIANT_CRAB said: sir sir Sir sir Sir

    Ron Paul.

    Thanked by 2trewq GIANT_CRAB
  • eLohkCalbeLohkCalb Member
    edited September 2014

    GoodHosting said: hard-coded default reaction is angry

    Not quite. The default reaction is actually, to troll.

    Edit: By the way, route limits and route mistakes are two different beasts, mixing beasts together will get you a Frankenstein.

    Thanked by 1Maounique
  • I'm a little puzzled by renesys's choice of words, mainly this: "causing disruptions to traffic in places as far-flung from the USA as Pakistan and Bulgaria."

    Is Bulgaria really seen as that very very very remote part of the internet?

  • rds100 said: Is Bulgaria really seen as that very very very remote part of the internet?

    from the USA is the key part of that sentence, I believe. Both pk and bg indeed are.

  • jarjar Patron Provider, Top Host, Veteran

    Mark my words. The greatest attack on the Internet has yet to come, and has only been avoided this long by its complexity.

    Thanked by 1Spencer
  • Jar said: Mark my words. The greatest attack on the Internet has yet to come, and has only been avoided this long by its complexity.

    I'm surprised there hasn't been more attacks targeted at root servers. I also find it odd how NASA, ARL and DISA operate so many of these servers.

  • @VolumeDrive

    You broke the interwebs.

    Thanked by 2Mark_R volumedrive
  • MaouniqueMaounique Host Rep, Veteran

    alexh said: I'm surprised

    Me too. Actually, there was a saying, if the building industry would work like the Internet industry, the first woodpecker flying by would destroy civilization.

  • due to unstable network, currently I don't use volumedrive.

  • Soon as this was brought to our attention, we promptly filtered the outbound announcement and removed Atrato from our peering to perform further testing.

    Last week we underwent a capacity upgrade on our 10G circuits with Atrato and during this changeover, they moved our peering POP from Ashburn to NYC (more capacity) and apparently the max_prefix filer wasn't copied from the Ashburn infrastructure to the NYC resulting in the leak.

Sign In or Register to comment.