New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Comments
okay...
My three VPS still does not work, and still no answer to my ticket
I do not have access to the panel, when I click I have this error message:
same here.
same here, i openned a ticket asking if this will be solved today and got no response.
sit and wait. it's midnight here, so who knows how long they can keep up with watching the progress or trying to fix things without getting some sleep. that much data will take time to rebuild/transfer/whatever.
would be very sad if to see this product range getting derranged somehow.
Shit happens.
I have a 1TB box hosted at ZXHost. Fortunately, it only hosts backups and I'm not too concerned apart from the fact I'd have to reupload everything.
Wasn't ZXHost operating with Proxmox? If so, that error could be as simple as Ashley replacing out failing hardware and fucking up the hosts file..
Has anyone had an update this morning?
I received an email stating that things were being brought back up. I did not have any problems when the issue first started, but now it's down and unable to boot. No biggie though, just a backup to a backupserver.
The "old infrastructure" was operated with Virtualizor. I do not know what sorage backend they used, but disk latency has always been very low. It was running amazing, great stuff.
The "new infrastructure" I was migrated to is Proxmox with Ceph (from what I know). The disk latency there was much worse, about 10-100x slower compared to the old one. And then some days ago the Ceph cluster encountered these problems we are talking about. As they mentioned in their mail they are hitting a bug in Ceph. Unfortunately there are no final fixes regarding this in the bugtracker yet. Might end up a little bit more complicated than just replacing hardware.
http://tracker.ceph.com/issues/21142
Nope
Update: "fyi, zxhost.co.uk runs on nginx on fedora"
Wow, it's getting better and better!
Checked
https://www.clouvider.co.uk ?
@Clouvider
They promise 100% uptime. I don't think the "noob host"-tag applies here though. I do not know in how far @Clouvider acctually keeps this promise but 100% SLA sometimes is meant to imply the idea of being eligable for a compensation for any outage of service..not necessarily 100% actual uptime. A positive step of the provider towards your expectations of an excellent service so to speak.
"Hello,
We have a work around that is allowing us to bring the OSD's online, we are down to the final OSD's now and expect to be able to start powering on VM's within the next few hours.
An update will be sent out once VM's start to boot.
Thanks,
ZXHost"
Fingers crossed.
why did they switch from 23media to ovh?
Luckily I only use this for cold storage but won't be renewing.
they didn't, at least not for all services. I have three services which are not from the storage range and not affected by all of this. still going strong in frankfurt/23media. yet the IPs for my storage VMs still route to frankfurt too, though at the moment those VMs of course are down as a result of the ceph disaster.
I don't see them changing anything right now but can't tell what future plans might look like or not or how they dealt with those migrations that obviously already have taken place before... it also could be a case by case thing, depending on the ressource usage and such? just a wild wild guess.
25 hours...
27 hours
they dont answer the tickets.
The suspense is killing me. Post updates every minute until they are back online please.
They have sent out status updates and probably have a tonne of tickets asking when it's going to be back online just be patient.
as your wish.
27 hours, 10 minutes, 43 seconds.
I totally understand your anger. no one here likes downtimes like this. but think about what you would like more to be done by them. fixing the issues so all can come back online or answering hundreds of tickets. even if you assume it's just a minute per ticket to select, read, put in some premade answers via copy'n'paste - 300 tickets makes it 5 man-hours just to handle that.
but be optimistic... it's friday evening, weekend ahead. go out and party, I am sure they will continue to fix issues early monday morning :-P
Yeah I got the same email update this morning. I'm not worried about the downtime if they're doing disk rebuilds or whatever. Rebuilds are slow and disk intensive and on cheap storage plans I'd frankly prefer having user vm's (including mine) shut down while they are happening, to decrease the likelihood of another crash during restore.
I notice that the emails have a lot of spelling errors. Does anyone remember if Ash's emails were like that before? I don't, so it makes me wonder if someone other than Ash is writing them.
I sent ZX an email saying data integrity is more important than uptime and that anyone using these cheap storage plans for critical services is nuts. Mine are cold backups. I do still wonder about the overall state of ZX, given that the 3 year 2TB offer was/is a little bit too attractive (though of course I bought it anyway). But we'll see.
@emptyPD So were you able to upload your clients backups to some temporary solution (like e.g. DO?)? Or did you not have like any offsite backups at all?
Hope it will all work out^^
And that's exactly why there is Twitter or their status page. Let's check them:
and
Yeah cool, nothing new for more than a day. I also didn't receive any mails. I don't know where you guys got your information from, but they maybe should put it on a public place. In times of such major incidents it is crucial for a good provider to keep posting updates. At least one line every hour does not steal anybodies time fixing the problem. Every hoster has their issues (even Amazon or OVH), the difference is how they handle it.
@AshleyUk I am disappointed. Not because you didn't answer my tickets for a week and not because the service is still down, but because I don't have any idea what happens and there is no way to find out. By the way, not only storage clients but also regular KVM clients are impacted by the current downtime. I run some productive semi-important shit on there and its always shitty to use slightly older backups, go on with them and merge everything in the end.
Funny enough I got the invoice email today
You expect them to update every hour? So you want, rebuilding, rebuilding, rebuilding, rebuilding.......
while I agree that a few more updates would be convenient, what would you expect them to write? "problem didn't change, still working on it" or more like "we're still here, not deadpooled yet"?
hope you get my point, I don't think updates on ongoing work won't help much if they can't tell how long it may take. other then earning another shitstorm if they can't keep that ETA... the rest is just about customers fear loosing things.
he hasn't been logged since Oct 19th...
that's odd, but are you sure it's related? my services I have outside the storage offers are working fine without any interruption