New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Comments
It would be nice of them to move their IRC server offsite, because now it's down as well and there's no good place to stay up to date. Well, at least there's this thread.
@Francisco
https://twitter.com/frantechca
Frantech Solutions @FrantechCA 14m
I'm aware, I think FH blew a router, just getting on the phone with them now
Frantech Solutions @FrantechCA 11m
@Uroboros_WoW No idea yet, I'm hoping it's something like a switch of theirs needs a restart. On hold waiting on Rob. Maybe it's just us.
FH had some electricion on site and supposedly they broke something.
I'm hoping it didn't bring our cage down and it's just our uplink but we'll see.
I'm going to yell at Don when I go down at the end of the month.
Grumble grumble annoying.
Francisco
They are aware of it:
https://twitter.com/FrantechCA
P.S. Late reply. ignore :-)
This is the second time in a month or so -- I almost feel like I am eating words about Fiberhub....they were pretty good for a few months, but they don't seem to be taking this power issue seriously.
Seems like lately, I am building systems to withstand more failure than I ever anticipated, with multi-dc being a requirement. My systems show uptime of about 26mins as of now...
I've been bugging Dilt and Fran about that for ages.
I have about 10 servers there, and they all lost power. They are back up now though.
Glad you're aware of it at least. I expect shit to come up from time to time, but this is the second time it happened today, which moves past isolated incidents and into "what the fuck is going on?" territory. The last time when things came back up there was no actual shutdown, just a connection interruption, so hopefully this will be the same way.
So did the router there just go down or power to the entire facility or just A leg power?
Folks need to audit the redundancy there.
Reminder to providers:
This is an example of why you have redundancy on your core customer systems (WHCMS, Solus Master, etc.). Ideally, offsite, off your locations, elsewhere.
Will be interesting to see how the router does on bringing another up.
my 2 LV VPS's are down, If this is another power outage not looking to good for fiberhub as its the second time within a month.
OVH in Canada is great on the routing since they peer and haul their own traffic. Some real low latency you typically don't see coast to coast in the US @Voss.
Fiberhub will be fine. That electrician however, someone needs to send him back for apprenticeship.
@pubcrawler
I think they might need to just call it a lose and find a new one that knows how to maintain power.
We definitely had power disruptions there as well.
Fiberhub seem to be looking more and more unreliable how long have you been with them and what kind of experience have you had with them? Just wondering if this is a ongoing issues.
Thanks
Jonathan
Look at the upside, I can test latency now with the link probably unused
Certainly way under capacity at point in LAX.
Looks like Vegas is back online.
Another provider there I just logged into. 2 minutes uptime. Yep total power loss probably facility wide.
Which means long fsck time ahead...
@pubcrawler @rds100
http://www.lowendtalk.com/discussion/comment/207044#Comment_207044
Yeah Fran said Fiberhub messed up yet again.
Yeppers @rds100.... My storage node with BuyVM is probably checking the bits right now.
I wouldn't want to be working the support desk this evening...
Their router just booted (prefix appeared in BGP). The nodes will probably take some time.
Mine just got back online:
They're all spinning pretty damn quick. We're already at 50% network utilization.
On the plus side, I think I debugged one of the last speed crap outs I was seeing. My home speeds bumped from ~400K/sec to like 4.5MB/sec in a single stream and many others have shown large jumps too.
I guess we'll see once the network is back at full usage and we get some tickets out.
Francisco
Francisco
Sighs. Got new LV backup server (KVM) - Waiting for it to spin up (or spin it up manually)
Just spin it. Anthony has pushed what starts he can but if you have an ISO that didn't quite detect right it'll stop the startup.
Francisco
Spun up manually a minute or so after the thread, watched it fix some journaling in VNC, back up
Hum, this is getting ridiculous. I mean, ok, not a top DC, however, 2 times losing power in a month that is more than just murphy.
Did EGI manage to hire someone as electrician named murphy at FH ? :P
Did FH go like this before buyvm moved ? We could guess if murphy was on site or came in the buyvm truck...