Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In with OpenID
Advertise on LowEndTalk.com

In this Discussion

Generator or Giant Battery Bank?
New on LowEndTalk? Please read our 'Community Rules' by clicking on it in the right menu!

Generator or Giant Battery Bank?

randvegetarandvegeta Member, Provider

Every few years or so, our building's power gets shut down for routine maintenance. We have a Diesel generator in place to handle such outages. Power in HK is incredibly stable and reliable, and other than the odd brown out during severe weather conditions, there hasn't been a single un-planned outage in over 6 years.

As such we don't use it much and it kind of gets a bit neglected. As long as it's working, it will likley stay in place, but I'm wondering weather or not generators still make sense in the future (of course some places will have more justification than others).

Wondering if large battery banks are starting to make sense. Batteries are getting denser and cheaper. Flow batteries also look very promising as they apparently do not degrade (ever) and have extremely deep cycles (can fully charge and discharge without any adverse effects).

Anyone using very large battery banks over generators? What's your capacity at full load (in hours)?

Would end users have any less confidence in a host that primarily relied on large battery banks rather than a traditional generator?

«1

Comments

  • lurchlurch Member

    The old DC's I used to work at had ups's with rows of batteries when power failed batteries took over and then the generator locked it keep them topped up while the ups smoothed the power delivery. There was also a backup generator around 2000kva. Not sure how long you would get just on modern batteries you would need to do some calculations.

  • raindog308raindog308 Moderator

    randvegeta said: Would end users have any less confidence in a host that primarily relied on large battery banks rather than a traditional generator?

    I have zero confidence with a host that cannot tell me what class their DC is.

    If you know what class your DC is or what class you want it to be, your questions are already answered.

    My Advice: VPS Advice

    For LET support, please click here.

    Thanked by 2Aidan doughmanes
  • teamaccteamacc Moderator

    Lets assume a 44U rack with 100w per 2u unit, with some spare room for switches etc, at 2KW per rack. Lets say you want to get the thing to run overnight (12h), that'd mean 24KWh. Lead acid batteries do about 100Wh per liter, so you'd need about 250L worth of space for batteries (per rack). A rack is about a meter in depth (give or take), and half a meter wide. This would mean you'd need about half a meter high worth of rack space to store enough batteries for a single rack. (around 12U lost, or a fifth of your total space is eaten by batteries).

    Same calulation for mass: lead-acid do about 40Wh/kg, so you'd need around 600kg worth of batteries. Do you really want to be lugging 600kg worth of batteries per rack every 5-10 year?

    And then for price: wikipedia lists them as "7-18 Wh/usd", so lets call that around 12 Wh/usd, or around 2000 usd "per rack you want to get power to"

    This ofcourse scales with battery density and "how long you want to be able to run it". Lead-acid are probably your best bet for cost-effectiveness (battery-wise) though.

    Ebay lists 20KW generators (enough for 10 racks) for anywhere between 500 usd (https://i.ebayimg.com/images/g/iT0AAOSw5ipaoGQs/s-l500.jpg , probably not the type you want), with "normal" pricing being beteen 5 and 10k, meaning the initial investment in a generator will be about a quarter to half of your initial investment in batteries, with the possibility to extend power delivery to your servers almost indefinitely (granted that you get someone to refuel the things)

    I'd say a generator is a pretty good thing to have for any outage that lasts for over 3 hours.

    Yo mama so fat each of her butt-cheeks has its own /8.

  • randvegetarandvegeta Member, Provider

    teamacc said: I'd say a generator is a pretty good thing to have for any outage that lasts for over 3 hours.

    Well we have the generator in HK as it is so for as long as the generator continues to work, we will likely not need to replace it.

    I'm thinking about new installations.

    But if we used my HK example, our generator can easily do 100KW. Using your estimated figures of 2KW per rack, that's 50 racks worth. Realistically though I'd say 25 racks to account for cooling, which in HK, you cannot shut off for any extended period of time.

    But let's say you have a 100KW load, and you want enough battery capacity for 12 hours. That would be 1,200KWH of batteries. No doubt that would be costly, you can pick up used EV batteries for less than $100 / KWH (usable). Probably doable for around $100k USD. And EV batteries are Li-Ion, so they are far denser, smaller and lighter than Lead Acid. I estimate it would take roughly the same amount of space as our generator (think 12x Tesla Battery Packs) currently uses. 1,200KWH worth of diesel translates to around 400 litres. That's obviously much less space/volume than 1,200kwh worth of batteries, but it's not nothing.

    Assuming you are comparing full load for 12 hours, and everything works, then the generator makes the most sense economically, by a wide margin, but it's not a particularly fair comparison.

    Batteries are more dynamic, and you can quite easily increase capacity as needed (space and budget permitting). Also, it is rare to need the full rated capacity of your generator, and it may be extremely rare in some places for power to be out for such extended periods of time that actually require such large capacity.

    Assuming you can source Li-Ion batteries at $100 /kwh, and each rack uses 2kw, and you need a 24kwh bank per rack, that would cost about $2,400. Which in my mind doesn't sound half bad. It would only use about 60L worth of space and weigh 160KG. Far less than Lead Acid.

    The Li-Ion batteries could last decades too, if maintained properly.

    I don't think $2,400 per rack is THAT much. The equipment that goes into a rack is considerably more. The ACTUAL size of the battery and inverter might be a bit of an issue, but I think the real question is whether or not a 12 hour at max load is a sufficient safety net for most customers? For context, in HK, I've never seen an unplanned outage EVER. And planned outages have rarely exceeded 2-3 hours.

    What do DC's do when their generators run out of Diesel? :D

    raindog308 said: I have zero confidence with a host that cannot tell me what class their DC is.

    If you know what class your DC is or what class you want it to be, your questions are already answered.

    Do you mean tier?

    I'm not sure why a battery bank could not replace a generator and still satisfy tier requirements.

  • randvegetarandvegeta Member, Provider

    Regarding my question about what happens when you run out of diesel, I do realize that some large DCs have contracts with providers that guarantee delivery of fuel in a crisis. And for very very large data centers using megawatts of power, that's very understandable since they essentially run a sizable power station. But for a small DC operation, if you can get a contract for a supplier to bring in diesel when needed, then it would also seem likely you can rent a generator in the event you may need more than 12 hours of power.

    The 12 hours of battery would then be sufficient to hold you over till the rented diesel generator arrives.

    In other parts of the world, the battery may be topped up by wind or solar :-).

  • Ifi read it correctly from your first answer you have done your calculation, if you think battery will suficient just do it, end user just care as long as their server worked

    Thanked by 1doghouch
  • FHRFHR Member, Provider

    randvegeta said: The Li-Ion batteries could last decades too, if maintained properly.

    Li-Ion batteries will not last decades if kept in fully charged state - which is exactly the state you want your batteries to be in for an always online backup power source.

    randvegeta said: you can pick up used EV batteries for less than $100 / KWH (usable)

    Tinkering with this in your home is fine, but I'm not sure I would want a DIY solution in a DC.

    Unlike Lead Acid batteries which will survive almost anything (and if they don't, no major event happens either way), Li-Ions require VERY careful monitoring and management.

    Each cell has to be monitored independently, as temperature and voltage are extremely critical. Balancing is also equally important (ensuring each cell is charged to the precisely same voltage as other cells).

    +100mV on a cell and you've got a miniature sun in your DC.

    Sure, these technical complications can be solved (look at Tesla Powerwall), but don't expect it to be as cheap as you envisioned.

    Affordable Semi-Dedicated VPS - Enjoy the performance to the fullest extent. | 40% OFF promo

    Thanked by 1Aidan
  • omelasomelas Member
    edited March 10

    I wouldn't care about battery backup, but if it's used EV batteries hacked together....

  • Regarding generators, note that there are different choices of fuel. Propane may be a choice, but natural gas might be available in the building and could thus provide continuous running without refueling. Also note that generators are normally inspected and tested by running for 30 minutes once a month which could be a consideration when deciding on what to use.

  • randvegetarandvegeta Member, Provider

    FHR said: Li-Ion batteries will not last decades if kept in fully charged state

    At 100%, true. At 80 - 90%, it should be fine. 50% would apparently be optimal for long term storage, but even at 90%, the life span (at least for Tesla batteries) seems to be pretty good.

    FHR said: Each cell has to be monitored independently, as temperature and voltage are extremely critical. Balancing is also equally important (ensuring each cell is charged to the precisely same voltage as other cells).

    Are you sure about this? This doesn't sound right. As far as I am aware, even Tesla's battery packs do not have individually monitored cells. Rather they are modularized.

    I may be wrong of course. I'm no expert. Where did you read this?

    FHR said: Sure, these technical complications can be solved (look at Tesla Powerwall), but don't expect it to be as cheap as you envisioned.

    Using new batteries will definitely be more costly, but battery prices are trending down any way, so it may not be too long before new batteries are $100 / kwh. Probably will reach this price point within the next 3 years or so.

    omelas said: but if it's used EV batteries hacked together.

    This is what the old EV batteries are destined to be. Battery packs may be retired from EVs as a result of degradation, but they may still retain 70-80% of their original capacity. They retire them from EVs because the reduced capacity reduced the vehicle's range, but for static storage, it really doesn't matter all that much. So life as static storage may exceed the time being an EV battery.

    As for being 'hacked' together, I don't see it as being any different to stringing together new batteries. It's not like I'm suggesting hacking together the UPS / Inverter side of things. Just talking about batteries here.

  • JSCLJSCL Member, Provider

    Generators are not cheap to run for long periods... battery backup would be better and any DC should really have both in place for UPS.

    I know that a DC here on the IOM is to be installing a bunch of Tesla Powerwalls to backup their new GPU environment... apparently that's quite a cost effective thing to do.

  • randvegeta said: This is what the old EV batteries are destined to be. Battery packs may be retired from EVs as a result of degradation, but they may still retain 70-80% of their original capacity. They retire them from EVs because the reduced capacity reduced the vehicle's range, but for static storage, it really doesn't matter all that much. So life as static storage may exceed the time being an EV battery.

    Can you get the appropriate electronics for those? I believe they're like 300 volts. Not 12, 24, or 48 volts. So are not going to work with standard converters.

  • randvegetarandvegeta Member, Provider

    Ole_Juul said: Can you get the appropriate electronics for those? I believe they're like 300 volts. Not 12, 24, or 48 volts. So are not going to work with standard converters.

    It's not that hard. There are a number of V2G systems already available. They tend to be limited to 6KW of output, so that may make a difference with regards to installation. I'm not sure how big these things are or if it is more cost effective to have many smaller systems or 1 large centralized system. But I don't think it make sense to think about things in terms of 12, 24 or 48 volts when dealing with Li-Ion. Most UPS sysems are designed to work with Lead Acid, and so the voltages and currents are all a bit off.

    JSCL said: Generators are not cheap to run for long periods

    Well fuel is expensive for sure but the up front capital is considerably less. Would need to consider the total cost of ownership over a given time-frame.

    As mentioned above, we have had to use our diesel generator only a couple of times, and each time, only for a few hours. We have tanks full of diesel that's just been sitting for over 5 years, un-used. Even if diesel was 10x the price, it would not be of any concern to us.

    I'm more concerned about ongoing maintenance and servicing, and the general life span of the generator, and when it comes to replacing it, which will make more sense.

    Ole_Juul said: Propane may be a choice, but natural gas might be available in the building and could thus provide continuous running without refueling.

    It's hard enough using Diesel, but gas/propane is not all that common in HK, so that I think would only add to the difficulties.

    From what I gather, no one is really using a pure battery back up solution.

    JSCL said: battery backup would be better and any DC should really have both in place for UPS.

    In our case we have multiple banks that can supply all equipment with enough power for upto 1 hour at full load (assuming the battery is in decent condition). The batteries get replaced every 5 years or so. After 5 years, those batteries would probably only last 15-20 minutes. Generator needs a couple of minutes to kick in if the power goes out. As the batteries age, they are really only good for handling brownouts, and super quick power maintenance work.

  • ClouviderClouvider Member, Provider
    edited March 10

    So you want to replace your generator with battery installation. What if your battery capacity is 4 hours and outage happens to last unusually long, say 3 days? You'll lose the load?

    Properly maintained generators (plural!) running in at least N+1 resilience, coupled with a decent fuel supply contract and plenty of capacity in the UPS system to keep the load while the generators are starting up is the insurance policy against the unusual problems.

    Clouvider Leading UK Cloud Hosting solution provider || UK Dedicated Servers Sale || Tasty KVM Slices || Latest LET Offer

    Web hosting in Cloud | SSD & SAS True Cloud VPS on OnApp | Private Cloud | Dedicated Servers | Colocation | Managed Services

    Thanked by 3kcaj MCHPhil lazyt
  • randvegetarandvegeta Member, Provider
    edited March 10

    Clouvider said: So you want to replace your generator with battery installation. What if your battery capacity is 4 hours and outage happens to last unusually long, say 3 days? You'll lose the load?

    As long as our generator works, there is no plan to replace. I'm thinking about the future, for future build outs or in the even the generator goes tits up.

    Where did you get 4 hours from? I've been talking about 12 hours at max load.

    As for an outage lasting 3 days, that's unheard of in HK. I reckon if such an outage ever occurred, there would be far greater problems than the downtime of a data center. But yes, downtime would be inevitable.

    Clouvider said: Properly maintained generators (plural!) running in at least N+1 resilience

    Yeah that's not really an option. Space is a bit of a premium in HK. Power + UPS + Generator is as good as it gets for a small Data Center over here. I'm quite certain most small DC's here don't even have the generator. The only alternative I can think of would be to rent an additional generator if and when needed. One that would fit on the back of a truck or in a trailer.

    I have some other ideas about doing things in Lithuania. A very large battery bank, supplemented with dual power feeds to the grid with solar and wind. But that's quite a bit more expensive.

  • kcajkcaj Member

    Have to echo what @Clouvider says. Regardless of the size of your battery install, it is a finite supply. When those batteries go flat, there is no 'battery supply' contract to call upon.

    With a diesel generator when your fuel supplies start to look low, you call your fuel supply contract and top it up. Run time is infinite.

    Thanked by 2Clouvider MCHPhil
  • kcajkcaj Member

    randvegeta said: Yeah that's not really an option. Space is a bit of a premium in HK

    If space is a premium and you really must squeeze on it - go for DRUPS.

    It's crazy to think that any data centre operator would ever consider moving from a power generating model to a power storage model as a mains failure contingency.

    It sounds like you've discovered some cow boys. Looking over the facility specifications for Equinix and Telehouse facilities in HK, they all have standby diesel generators.. just as you'd expect.

    Thanked by 1Clouvider
  • randvegetarandvegeta Member, Provider

    kcaj said: It's crazy to think that any data centre operator would ever consider moving from a power generating model to a power storage model as a mains failure contingency.

    You must have missed the part where I mentioned you could simply rent a generator on the same basis that you can get a contract for a supply of fuel for a generator.

    Storage of energy is storage of energy. How much Diesel can be stored on site is also finite. Relying on a 3rd party to supply diesel, is not much different, in my mind, to having a contract from a supplier who also provides a suitably sized generator.

    Am I missing something here?

    kcaj said: It sounds like you've discovered some cow boys.

    There are quite a lot of small providers, but I don't think it would be fair to refer to them as 'cowboys'. Not every company has US$100M to spend on building a DC.

  • FHRFHR Member, Provider

    randvegeta said: FHR said: Each cell has to be monitored independently, as temperature and voltage are extremely critical. Balancing is also equally important (ensuring each cell is charged to the precisely same voltage as other cells).

    Are you sure about this? This doesn't sound right. As far as I am aware, even Tesla's battery packs do not have individually monitored cells. Rather they are modularized.

    I may be wrong of course. I'm no expert. Where did you read this?

    Yes, I'm sure.

    Regarding temperature control, Tesla's battery packs are water-cooled, so they don't have to monitor the temperature of each cell but just the coolant.

    Regarding voltage monitoring and balancing, you don't have to monitor each cell, but rather an each group of cells.

    Tesla 85kWh battery pack is composed of 7104 cells in 16 modules. Each module contains 6 groups of batteries in series and each group is composed of 74 cells in parallel.
    In technical notation, the battery pack designation is "96S74P" - 96 groups in Series with each group having 74 Parallel cells.

    Each of these 96 groups is individually monitored and controlled by a BMS - battery management system.

    There's a one BMS per module, each servicing 6 groups (you can see the "CELL 0 - 6" connector on the right side of the board). So 16 BMS modules per the whole battery pack.

    This BMS ensures that each group of cells is charged to the precisely same voltage - and if it isn't, it will try to balance them by discharging the overcharged ones.

    Source - teardown pictures: https://teslamotorsclub.com/tmc/threads/pics-info-inside-the-battery-pack.34934/

    It works exactly like a laptop battery (which are usually in 3S1P or 3S2P configuration - and yes, you will find a BMS in each laptop), just on a much larger scale.

    Affordable Semi-Dedicated VPS - Enjoy the performance to the fullest extent. | 40% OFF promo

  • randvegetarandvegeta Member, Provider

    FHR said: Yes, I'm sure.

    Regarding temperature control, Tesla's battery packs are water-cooled, so they don't have to monitor the temperature of each cell but just the coolant.

    Ahh fair enough. Well I was thinking about using old Tesla batteries any way. If we could obtain the whole pack, then perhaps we could use the same BMS and cooling systems. Though that may be more difficult than it sounds.

  • AidanAidan Member

    Insurance can get a bit dicey if you're stacking that many batteries, lead-acid will require proper ventilation & any type of lithium will require a proper fire-control system.

    Batteries are meant to keep you up long enough to switch to a generator/2nd power line.

    Thanked by 1Clouvider
  • randvegetarandvegeta Member, Provider

    Aidan said: Insurance can get a bit dicey if you're stacking that many batteries, lead-acid will require proper ventilation & any type of lithium will require a proper fire-control system.

    Good point about insurance.

  • raindog308raindog308 Moderator

    randvegeta said: Do you mean tier?

    I'm not sure why a battery bank could not replace a generator and still satisfy tier requirements.

    Yes: tier, class - different terms for same thing.

    I was under the impression that the different levels mandated specifics in terms of power (i.e., generator/battery mix), but perhaps they only mandate experience. I'm not a DC guy, but I interact with them and all the pros I've dealt can rattle off their datacenter certifications and ratings. If you're working on a serious DC, I'd try to align with industry standards.

    My Advice: VPS Advice

    For LET support, please click here.

  • teamaccteamacc Moderator

    @randvegeta said:

    kcaj said: It's crazy to think that any data centre operator would ever consider moving from a power generating model to a power storage model as a mains failure contingency.

    You must have missed the part where I mentioned you could simply rent a generator on the same basis that you can get a contract for a supply of fuel for a generator.

    When would you start calling a generator company for that though? Immediately at the outage? When your batteries are halfway through? (6h seems like quite short-notice) When your batteries are 2h away from dying?

    Yo mama so fat each of her butt-cheeks has its own /8.

  • raindog308raindog308 Moderator

    randvegeta said: Storage of energy is storage of energy. How much Diesel can be stored on site is also finite. Relying on a 3rd party to supply diesel, is not much different, in my mind, to having a contract from a supplier who also provides a suitably sized generator.

    I was curious...

    https://en.wikipedia.org/wiki/Energy_density

    You can store a lot more energy in Diesel than you can in Lithium Ion in the same size of space. Even with the generator to convert it to usable electricity, I suspect that diesel will win any energy density comparison.

    Of course, for maximum efficiency you should use deuterium, which is five orders of magnitude more energy dense than diesel. Then if your mains fail, you just need to touch off a fission bomb to kick start it.

    My Advice: VPS Advice

    For LET support, please click here.

  • ClouviderClouvider Member, Provider
    edited March 10

    Who would deliver generators up to specs and guarantee it, and install it in time when half of the town has no power ?

    Where would you put it and the tanks if you have no space ?

    This is fantasy world.

    Clouvider Leading UK Cloud Hosting solution provider || UK Dedicated Servers Sale || Tasty KVM Slices || Latest LET Offer

    Web hosting in Cloud | SSD & SAS True Cloud VPS on OnApp | Private Cloud | Dedicated Servers | Colocation | Managed Services

    Thanked by 1kcaj
  • doghouchdoghouch Member
    edited March 10

    @raindog308 said:

    randvegeta said: Storage of energy is storage of energy. How much Diesel can be stored on site is also finite. Relying on a 3rd party to supply diesel, is not much different, in my mind, to having a contract from a supplier who also provides a suitably sized generator.

    I was curious...

    https://en.wikipedia.org/wiki/Energy_density

    You can store a lot more energy in Diesel than you can in Lithium Ion in the same size of space. Even with the generator to convert it to usable electricity, I suspect that diesel will win any energy density comparison.

    Of course, for maximum efficiency you should use deuterium, which is five orders of magnitude more energy dense than diesel. Then if your mains fail, you just need to touch off a fission bomb to kick start it.

    Why not buy a CANDU reactor? I mean, it’ll only set you back a few...errr...hundred million, but you’ll have power to weather through any outage.

    Edit: an added bonus is the radioactive waste it generates :]

    Why not check my site (https://dawgy.pw) out? If you need to upload a few GIFs, give https://i.dawgy.pw a try <3

    Thanked by 1raindog308
  • raindog308raindog308 Moderator
    edited March 10

    doghouch said: Why not buy a CANDU reactor? I mean, it’ll only set you back a few...errr...hundred million, but you’ll have power to weather through any outage.

    I read a little further down in that Wikipedia article and would like to present an improved proposal.

    Antimatter. 100% energy efficiency. No fission bomb needed, and no radioactive waste.

    True, a gram is estimated to cost $62.5 trillion to produce, which is nearly the entire annual global GDP. But with a few black friday promos...

    My Advice: VPS Advice

    For LET support, please click here.

    Thanked by 3MCHPhil FHR doghouch
  • randvegetarandvegeta Member, Provider

    Clouvider said: Who would deliver generators up to specs and guarantee it, and install it in time when half of the town has no power ?

    Where would you put it and the tanks if you have no space ?

    This is fantasy world.

    LOL. You're unbelievable selective in what you think contractors can and cannot fulfil.

    So on the one hand you think that a contract for fuel supply is perfectly doable, but not a portable generator? Now obviously I'm not talking about a multi-megawatt generator used by behemoth data centers, which obviously cannot be easily transported. But a 100KW generator, which is more than ample for most small DCs, which is what I'm talking about here, can quite easily fit in a trailer. Cabling for 100KW is also not so difficult.

    If expecting a company to supply a generator is living in a fantasy land, then so too is it a fantasy that a company can deliver ample supply of fuel.

    There are plenty of companies that lease out generators. And there would be no reason why they would not be able to deliver the generator on demand in the same way fuel is delivered. Should there be a road block or anything like that then delivery of fuel is also affected.

    I don't think you can have it both ways.

    teamacc said: When would you start calling a generator company for that though? Immediately at the outage?

    Do most contracts for fuel supply not include some guaranteed delivery time? It's very much the same question as for diesel. When do you call for a refill?

    If the SLA for fuel delivery is say... 4 hours, I'd say you should probably call when you have around 6-8 hours remaining. Or indeed just have it on stand-by for scheduled maintenance work that may exceed the capacity of your energy storage.

    raindog308 said: You can store a lot more energy in Diesel than you can in Lithium Ion in the same size of space. Even with the generator to convert it to usable electricity, I suspect that diesel will win any energy density comparison.

    Yes the energy density of Diesel is far far greater than any battery storage. It does not diesel can pretty much last forever too (doesn't degrade over time or go bad), and is not flammable, so storage wise is safer. But generators do take up quite a lot of space, and the maintenance and servicing also need to be factored in.

    Again, this is just an alternative to having on site generators. Traditionally, battery storage has been very expensive, but the cost is falling dramatically, and the typical outage in the developed world tends to be incredibly short. Of course having a large battery bank that can last a DC for 12 hours would be much more costly than having a generator with fuel that can last 120 hours, but you need to over provision power with a generator, whereas you can expand your battery capacity over time, so I think there may be some cost benefits.

    I think with new high capacity battery banks, it opens up new possibilities for how data centers can handle redundancy, and at scale. For example, you could start with a smallish 6KW UPS with 72KWH battery, which would not cost that much (less than $10K), and possibly have a contract for a fuel and generator supply for that level of load. As your load increases, your contract will grow too, so you need not necessarily worry about the infrastructure.

    This is all just hypothetical of course, since batteries are still kind of expensive, and the prices I am mentioned are for 2nd hand, and certainly the availability of second hand batteries could could not satisfy the demand of data centers. But battery production is ramping up at an incredible rate, and prices are falling just as fast. Is such a scenario truly just a fantasy?

  • ClouviderClouvider Member, Provider

    To mimic your response style: “LOL!” Certainly, because pouring gas into the tank is significantly simpler process than commissioning a large generator.

    What are you even talking about here.

    Clouvider Leading UK Cloud Hosting solution provider || UK Dedicated Servers Sale || Tasty KVM Slices || Latest LET Offer

    Web hosting in Cloud | SSD & SAS True Cloud VPS on OnApp | Private Cloud | Dedicated Servers | Colocation | Managed Services

    Thanked by 1kcaj
  • randvegetarandvegeta Member, Provider

    Clouvider said: To mimic your response style: “LOL!” Certainly, because pouring gas into the tank is significantly simpler process than commissioning a large generator.

    What are you even talking about here.

    I don't know about that. It seems fairly simple to rent a generator. I've never had to rent a generator in HK but in Lithuania we rented a very large generator for a few days. Was very cost effective, and very easy. Running power cables was really not an issue.

    Are you seriously saying that it's difficult to run a few cables and plug them in?

  • AidanAidan Member

    randvegeta said: 100KW generator, which is more than ample for most small DCs, which is what I'm talking about here, can quite easily fit in a trailer

    If you can find a local company that's got a good reputation, I guess you can do that. An empty 100kW diesel generator shouldn't weigh more than ~1.7tons so it's possible, I've rented a couple of 200kW towables for construction projects - but standby fees will most likely be rather high.

    randvegeta said: If the SLA for fuel delivery is say... 4 hours

    That's a bit optimistic, on this small scale you'll most likely arrange it through the people who'll bring you a generator.

    Thanked by 1Clouvider
  • randvegetarandvegeta Member, Provider
    edited March 10

    Aidan said: That's a bit optimistic, on this small scale you'll most likely arrange it through the people who'll bring you a generator.

    In Lithuania, where our DC is, there is a place just a couple hundred meters away where we can rent industrial equipment, including large power generators. It was very inexpensive and fast to arrive, but mainly because it was basically next door. You can even rent cranes from those guys.

    I'm not sure what the situation would be in HK though.

    Thanked by 1Aidan
  • lazytlazyt Member

    Just remember there are events where fuel can not be delivered. Or where the tanks end up under water thus unusable. Just look at when Sandy hit the US coast.

    Have I mentioned how much I hate auto correct recently?

  • ClouviderClouvider Member, Provider

    @lazyt said: Just remember there are events where fuel can not be delivered. Or where the tanks end up under water thus unusable. Just look at when Sandy hit the US coast.

    Aye, no one is saying that using additional backup methods is bad. We're merely pointing out that replacing generators with batteries is not delivering minimum resiliency standards expected in today's datacentres.

    Clouvider Leading UK Cloud Hosting solution provider || UK Dedicated Servers Sale || Tasty KVM Slices || Latest LET Offer

    Web hosting in Cloud | SSD & SAS True Cloud VPS on OnApp | Private Cloud | Dedicated Servers | Colocation | Managed Services

  • randvegetarandvegeta Member, Provider

    @Clouvider said:

    @lazyt said: Just remember there are events where fuel can not be delivered. Or where the tanks end up under water thus unusable. Just look at when Sandy hit the US coast.

    Aye, no one is saying that using additional backup methods is bad. We're merely pointing out that replacing generators with batteries is not delivering minimum resiliency standards expected in today's datacentres.

    What happens when your generator runs out of the 4 hours of fuel you keep on site and the contractor charged with supplying you more fuel can't get to you for whatever reason?

    Or what happens if you have your fuel and your datacentre has plenty of power but your upstreams don't have the appropriate power resiliency because the whole city is out of power and they can't get to their pops to hook up a generator.

    This is where sattellite connectivity comes in handy. Not that any of this matters, since a city wise outage probably falls under force majeure.

    There's been times entire countries have been knocked offline because of a single undersea cable being cut or damaged.

    There's always something. Always a possibility that something may happen that knocks you offline. It's happened in both HK and LT that our upstreams went down when the power in our building was down for maintenance because their UPS were insufficient for the duration of downtime. To hell with the SLAs! That didn't mean didly squat.

    Now we maintain a couple of wireless links in both locations for emergencies. Latency would go through the roof and performance would crawl, but at least things would remain online.

    But seriously, if there was an excessivly long downtime of power in HK, that would be pretty serious. Genuinely I think that would fall under force majeure.

  • AidanAidan Member

    randvegeta said: What happens when your generator runs out of the 4 hours of fuel you keep on site and the contractor charged with supplying you more fuel can't get to you for whatever reason?

    You get a jerry can & go buy some more? ;-;

  • randvegetarandvegeta Member, Provider

    @Aidan said:

    randvegeta said: What happens when your generator runs out of the 4 hours of fuel you keep on site and the contractor charged with supplying you more fuel can't get to you for whatever reason?

    You get a jerry can & go buy some more? ;-;

    Do pumps still work during power outages?

    That's actually an option. There's a petrol station a few hundred meters away and another less than a KM away.

    The only problem is you're not supposed to use Gerry Cans in HK, and only commercial vehicles are supposed to be able to use diesel.

  • AidanAidan Member
    edited March 10

    randvegeta said: Do pumps still work during power outages?

    Would be best to ask the individual petrol station, most modern pumps require power, though most decent sized stations will include a small generator to run the pumps.

    randvegeta said: The only problem is you're not supposed to use Gerry Cans in HK, and only commercial vehicles are supposed to be able to use diesel.

    I'm sure that a blackout would count as extraordinary circumstances.

    edit: Whilst you're there, also inquire what's the largest size container that they'll fill & if plastic/metal is okay - some places can be rather picky.

  • erwinerwin Member

    @Clouvider said: Who would deliver generators up to specs and guarantee it, and install it in time when half of the town has no power ?

    Where would you put it and the tanks if you have no space ?

    This is fantasy world.

    You don't need a fantasy world to get this done.... for sure its depending on a lot of things, but a proper energy supplier should be able to do it in a short time. they know the exact reason for the outage and average repair time... the generators should be up and running at your site in 2-4 hours in my experience.

    Thanked by 1randvegeta
  • ClouviderClouvider Member, Provider

    @erwin said:

    @Clouvider said: Who would deliver generators up to specs and guarantee it, and install it in time when half of the town has no power ?

    Where would you put it and the tanks if you have no space ?

    This is fantasy world.

    You don't need a fantasy world to get this done.... for sure its depending on a lot of things, but a proper energy supplier should be able to do it in a short time. they know the exact reason for the outage and average repair time... the generators should be up and running at your site in 2-4 hours in my experience.

    Well, not without expecting the silence of the lost load you can’t be sure of it when half of town has no power on Sunday night.

    2-4h is tight SLA for the petrol to reach your site, yet alone generator being guaranteed to be comissioned and ready for you.

    Shorter SLA is potentially possible in certain cases but would involve very steep costs making the whole venture of replacing it with the batteries non-economical even if no outage actually happens.

    Clouvider Leading UK Cloud Hosting solution provider || UK Dedicated Servers Sale || Tasty KVM Slices || Latest LET Offer

    Web hosting in Cloud | SSD & SAS True Cloud VPS on OnApp | Private Cloud | Dedicated Servers | Colocation | Managed Services

  • randvegeta said: As for an outage lasting 3 days, that's unheard of in HK.

    I thought it was the same in New York. Until hurricane Sandy when everybody was out for at least 2 days and many admins couldn't even get to work and had to do what they could remotely - which only goes so far when it's a server power issue.

    The weather isn't as predictable as one might like. Here's a list of major outages. HK, just might be next.

    Thanked by 1Clouvider
  • omelasomelas Member

    Wouldn't upstream ISP go offline at that long power outage?

  • omelas said: Wouldn't upstream ISP go offline at that long power outage?

    Major exchanges usually aren't using old car batteries.

    Thanked by 2Clouvider MCHPhil
  • omelasomelas Member

    @Ole_Juul said:

    omelas said: Wouldn't upstream ISP go offline at that long power outage?

    Major exchanges usually aren't using old car batteries.

    well, you mentioned hurricane Sandy but I dount prepering your datacenter for it would be helpful because nobody else expect it so your datacenter will be offline anyway...

  • Ole_JuulOle_Juul Member
    edited March 11

    omelas said: well, you mentioned hurricane Sandy but I dount prepering your datacenter for it would be helpful because nobody else expect it so your datacenter will be offline anyway...

    I believe some places remained open. And the ones that didn't have since taken appropriate measures. Telengy, a NY CLEC, was one such place.

    Edit to add. I just did a bit of a search, and during Hurricane Sandy when the power grid was down, I found this quote about Telehouse (where NYIIX is hosted)

    Carrier-neutral colo provider Telehouse America said Tuesday that its New York City facilities engaged backup diesel power ahead of the grid shutdown and that its Manhattan sites, 25 Broadway and Chelsea Center, were not operationally affected by the flooding.

    Thanked by 2omelas Clouvider
  • ClouviderClouvider Member, Provider

    Exactly. Plan for the worst, hope for the best.

    Clouvider Leading UK Cloud Hosting solution provider || UK Dedicated Servers Sale || Tasty KVM Slices || Latest LET Offer

    Web hosting in Cloud | SSD & SAS True Cloud VPS on OnApp | Private Cloud | Dedicated Servers | Colocation | Managed Services

    Thanked by 2Aidan MCHPhil
  • randvegetarandvegeta Member, Provider

    @Ole_Juul said:

    omelas said: Wouldn't upstream ISP go offline at that long power outage?

    Major exchanges usually aren't using old car batteries.

    That implies lead acid starter batteries. Li-ion is quite a bit different. Even if second hand, they are far superior to brand new lead acid ones.

    @Ole_Juul said:

    randvegeta said: As for an outage lasting 3 days, that's unheard of in HK.

    I thought it was the same in New York. Until hurricane Sandy when everybody was out for at least 2 days and many admins couldn't even get to work and had to do what they could remotely - which only goes so far when it's a server power issue.

    The weather isn't as predictable as one might like. Here's a list of major outages. HK, just might be next.

    New York has had many many many outages. Just look at that list and see how often things go down.

    And other than multi-million dollar DCs, if I'm not mistaken, just about everyone was affected in some form or another.

    Thanked by 1Ole_Juul
  • randvegetarandvegeta Member, Provider

    raindog308 said: Yes: tier, class - different terms for same thing.

    I was under the impression that the different levels mandated specifics in terms of power (i.e., generator/battery mix), but perhaps they only mandate experience. I'm not a DC guy, but I interact with them and all the pros I've dealt can rattle off their datacenter certifications and ratings. If you're working on a serious DC, I'd try to align with industry standards.

    There is no specific requirement to be able to generate power on-site for even tier 4 classification.

    I believe there is a certain amount of time the DC must be able to support for total power outage, which is normally through a Diesel generator. Most DC's fulfill redundancy requirements by having multiple power feeds from multiple providers from different substations. We have this in Lithuania, so the chances of both our power feeds going down is very small. The whole power grid would likely have to go down for both our sources to go down. I mean, of course fluke accidents happen, but generally speaking, the dual feed is the primary requirement.

    Onsite, you can fulfill the minimum time frame of onsite power via whatever means you have available. Traditionally this has been some sort of generator as it has been the most cost effective solution. This is still generally the case, but you could feasibly use batteries to also fulfill this requirement.

    You can get a 100KW generator for probably 10-20K (used) and the fuel can be seen as almost free. A 100KWH batter pack would also cost about 10-20K, but of course that only supplies 100KW for 1 hour, so if you want 24 hours on site power, the battery would optimistically cost 24x more than the generator. But being able to build out the battery bank in stages means the upfront cost may not be so large. As mentioned above, it could be as little $2,400 per rack for 12 hours capacity. It's still expensive but it's not unreasonably so. If you fit 16 servers in a 42U rack (as we do), that's only an extra $150 per server. Or $300 for 24 hours. So on a per server basis, it's really not all that expensive.

    Not every DC is built on a $100M budget, and getting all the perks of a tier 4 DC adds some serious cost. I don't think any provider here can claim they comply with tier 4 requirements. I would imagine the majority barely meet tier 2 requirements to be honest (but that's just a guess). That's not a criticism mind you. At LET level pricing, honestly any redundancy seems like it should be a bonus. No one would invest in all that redundancy for LET level services. To expect otherwise really would be living in a fantasy world.

    Thanked by 1Ole_Juul
Sign In or Register to comment.