Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


OVH Strasbourg Power Failure
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

OVH Strasbourg Power Failure

exception0x876exception0x876 Member, Host Rep, LIR

In case anyone wonders why his server is down - http://travaux.ovh.net/?do=details&id=28240

«13456711

Comments

  • You beat me to it :)

  • MikeAMikeA Member, Patron Provider
    edited November 2017

    I'm glad all of my servers are in GRA.

  • ClouviderClouvider Member, Patron Provider

    How you can loose power to the entire Datacentre ? They have one feed only ? If so, then wow.

  • randvegetarandvegeta Member, Host Rep

    @Clouvider said:
    How you can loose power to the entire Datacentre ? They have one feed only ? If so, then wow.

    1 feed and no generator apparently.

  • Vova1234Vova1234 Member, Patron Provider
    edited November 2017

    Yes. This is the first time such a thing has been noticed.

    In SBG a large number of gaming servers i7-6700k, i7-7700k, i7-4790k.

    Now a huge number of game servers went offline.

  • No point in guessing at this point. The RFO should clear stuff up if there will be any.

  • MikeAMikeA Member, Patron Provider
    edited November 2017

    @xaoc said:
    No point in guessing at this point. The RFO should clear stuff up if there will be any.

    It will likely be next month, still waiting for SLA (+RFO) e-mail from last months outage in different DC.

  • ClouviderClouvider Member, Patron Provider

    @MikeA said:

    @xaoc said:
    No point in guessing at this point. The RFO should clear stuff up if there will be any.

    It will likely be next month, still waiting for SLA (+RFO) e-mail from last months outage in different DC.

    Wow. In contrast I receive my RFOs in relation to any Equinix or Virtus incidents in less then a week. I don’t even need to ask for it.

  • rm_rm_ IPv6 Advocate, Veteran

    Somehow they managed to fuck it up so hard, RBX just went down as well.

  • ClouviderClouvider Member, Patron Provider

    @randvegeta said:

    @Clouvider said:
    How you can loose power to the entire Datacentre ? They have one feed only ? If so, then wow.

    1 feed and no generator apparently.

    Any source in their website for this ?
    I’d love to point this to the next person who argues prices on OVH comparable services

    Thanked by 2jh_aurologic steny
  • exception0x876exception0x876 Member, Host Rep, LIR

    @rm_ said:
    Somehow they managed to fuck it up so hard, RBX just went down as well.

    Yes, RBX went dark for me too.

  • ProfforgProfforg Member
    edited November 2017

    Does someone see issues in GRA and RBX as well? I have most server down in all locations except Canada. So all in France.
    Do they have power issues on three geo-locations in France? Probably something with whole France?

  • MikeAMikeA Member, Patron Provider

    @Clouvider said:

    @MikeA said:

    @xaoc said:
    No point in guessing at this point. The RFO should clear stuff up if there will be any.

    It will likely be next month, still waiting for SLA (+RFO) e-mail from last months outage in different DC.

    Wow. In contrast I receive my RFOs in relation to any Equinix or Virtus incidents in less then a week. I don’t even need to ask for it.

    Yeah. Others I've used in the past were always within a few days, nowhere as large as OVH, but nothing small. It is a bit upsetting, maybe I should buy stocks from a DDoS appliance vendor..

  • rm_rm_ IPv6 Advocate, Veteran

    randvegeta said: 1 feed and no generator apparently.

    Their website said, I quote: "generators did not start". The incident website is mostly down now as well, so can't provide the exact link.

  • @Profforg said:
    Does someone see issues in GRA and RBX as well? I have most server down in all locations except Canada. So all in France.
    Do they have power issues on three geo-location in France? Probably something with whole France?

    Just got issues in RBX, judging from twitter it appears GRA and RBX is down now too.

  • MikeAMikeA Member, Patron Provider

    @Makenai said:

    @Profforg said:
    Does someone see issues in GRA and RBX as well? I have most server down in all locations except Canada. So all in France.
    Do they have power issues on three geo-location in France? Probably something with whole France?

    Just got issues in RBX, judging from twitter it appears GRA and RBX is down now too.

    Nothing of mine in Gravelines is down, doubt it'll be affected, it usually isn't when there's big problems in SBG/RBX in the past.

  • @rm_ said:
    Somehow they managed to fuck it up so hard, RBX just went down as well.

    I read this thread while eating and said " I don't care". Then I got email saying my servers are down.

  • ClouviderClouvider Member, Patron Provider
    edited November 2017

    @rm_ said:

    randvegeta said: 1 feed and no generator apparently.

    Their website said, I quote: "generators did not start". The incident website is mostly down now as well, so can't provide the exact link.

    How can generators not start ?
    Industry standard is at lest N+1 on the control system, N+1 in the actual generators and N+1 on the actual UPS system. On top of that majority of DCs offer you feed from at least 2 different UPS systems and have at least 2 as diverse as possible feeds out of public network. Besides they should be keeping enough juice in the UPS systems to be able to go there and run them manually in case of such an event. For example, Virtus currently keeps between 30-50 minutes depending on the system.

    I can find nothing about dual feed there, so it’s probably safe to assume there’s one feed, powered by a set of generators that apparently got screwed up at some point of either design or maintenance.

    Cost cutting exercise must have backfired.

    All this while increasing prices.

    Thanked by 1jh_aurologic
  • RhysRhys Member, Host Rep
    edited November 2017

    Forgot to steal money from the banks to pay the energy bill.

  • ClouviderClouvider Member, Patron Provider
    edited November 2017

    @hammad said:

    When was the latest generators run then ? They don’t test it ? Seems unlikely.

  • @hammad said:

    Google translate:

    We have a problem feeding SBG1 / SBG4. The 2 electric EDF arrivals are down (!!) and the 2 generator sets have failed (!!!). The set of 4 arrivals elec no longer feed the routing room. We are all on the problem.

  • KuJoeKuJoe Member, Host Rep

    I got an alert for my Kimsufi server in RBX and thought my payment didn't go through. I logged into their control panel and it says I don't have service with them but it shows all of my invoices paid. What kind of system half works during an outage? I would have felt better if the whole thing was offline instead of appearing to work and telling me all of my services are gone. Glad everybody here is on top of things because their status page isn't loading for me. :)

  • karanchookaranchoo Member
    edited November 2017

    shit got real , all 8 servers with them are down.

  • rm_ said: Their website said, I quote: "generators did not start"

    Sounds like something you would write on a tombstone :)

    Clouvider said: How can generators not start ?

    My friend once had a power outage in the lab he's managing, the power generator cough fire while starting. So I guess it's possible.

    Thanked by 1Aidan
  • datanoisedatanoise Member
    edited November 2017

    Clouvider said: Cost cutting exercise must have backfired.

    All this while increasing prices.

    Fuck yeah, this is impressive. It might hurt them pretty deeply. Except if well, you know, people don't care as long as it cheap.

    I guess it will depends how long it takes for them to fix the downtime.

    That's weird: why did the second location got down too?
    edit: routing issue on TH2. wow

  • jackbjackb Member, Host Rep

    @Clouvider said:

    @rm_ said:

    randvegeta said: 1 feed and no generator apparently.

    Their website said, I quote: "generators did not start". The incident website is mostly down now as well, so can't provide the exact link.

    How can generators not start ?
    Industry standard is at lest N+1 on the control system, N+1 in the actual generators and N+1 on the actual UPS system. On top of that majority of DCs offer you feed from at least 2 different UPS systems and have at least 2 as diverse as possible feeds out of public network. Besides they should be keeping enough juice in the UPS systems to be able to go there and run them manually in case of such an event. For example, Virtus currently keeps between 30-50 minutes depending on the system.

    I can find nothing about dual feed there, so it’s probably safe to assume there’s one feed, powered by a set of generators that apparently got screwed up at some point of either design or maintenance.

    Cost cutting exercise must have backfired.

    All this while increasing prices.

    I don't think OVH do dual feed servers, so they probably didn't bother with the rest of the infrastructure (except Colo sections perhaps - if they do Colo)

  • It's great to have multiple locations if a power outage in one affects the others as well

    Thanked by 1southy
  • randvegetarandvegeta Member, Host Rep

    @Clouvider said:

    @randvegeta said:

    @Clouvider said:
    How you can loose power to the entire Datacentre ? They have one feed only ? If so, then wow.

    1 feed and no generator apparently.

    Any source in their website for this ?
    I’d love to point this to the next person who argues prices on OVH comparable services

    Having generators without fuel, or that don't work is the same as not having a generator. The fact they are down means they do not have this contingency in place.

    Jesus, even our tiny datacentres in HK and LT both have dual feeds, 1hr UPS and backup generators. No wonder OVH can do rock bottom pricing. They basically just have a ton with normal power and an internet connection. That's not hard!

    Thanked by 2Clouvider joeri
  • KuJoeKuJoe Member, Host Rep

    randvegeta said: Having generators without fuel, or that don't work is the same as not having a generator.

    If you have a generator, it is not the same as not having a generator. :)

Sign In or Register to comment.