Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Advertise on LowEndTalk.com

LEGACY-03.LV.BUYVM.NET is down now.
New on LowEndTalk? Please read our 'Community Rules' by clicking on it in the right menu!

LEGACY-03.LV.BUYVM.NET is down now.

ersiteersite Member
edited December 2018 in Outages

Hi!
My vps is down over 24 hours since yesterday. I have submitted a ticket to buyvm but no one replies me. Does anyone have the same issue? I cannot boot up my vps!! It is very bad news during the Xmas holiday.

«1

Comments

  • You're losing millions. We're all sorry for your loss.

  • Don't run production stuff on cheap VPS.

  • Horrible news, let's tag @Francisco!

    Me, myself and a lowest common denominator named Bob - We all ❤️ @eol.

  • ersite said: My vps is down over 24 hours since yesterday. I have submitted a ticket to buyvm but no one replies me. Does anyone have the same issue? I cannot boot up my vps!! It is very bad news during the Xmas holiday.

    Do you have backups? If yes, spin a temp vps on a service like DO or Scaleway and be online in minutes until the issue being resolved, to avoid loosing millions due to downtime.
    If you don't have backups, learn your lesson and keep backups. No infrastructure stays for ever.

    Demanding immediate response on a Sunday 3 days before Christmas for a ~$5 vps and opening a bash thread as your first and only "contribution" on LET, without even backing up your claims (e.g. issue to your own vps, not the node itself), won't give you any sympathy here.

    Thanked by 2eol coreflux
  • FranciscoFrancisco Top Provider

    @ersite said:
    Hi!
    My vps is down over 24 hours since yesterday. I have submitted a ticket to buyvm but no one replies me. Does anyone have the same issue? I cannot boot up my vps!! It is very bad news during the Xmas holiday.

    Do you have a ticket?

    We've been fixing up anyone that got banged up by the power outage that hit the storage rack.

    Some VM's did get hosed so we've done credits, rebuilds, etc.

    Francisco

    Thanked by 2kkrajk coreflux
    BuyVM - Free DirectAdmin, Softaculous, & Blesta! / Anycast Support! / Windows 2008, 2012, & 2016! / Unmetered Bandwidth!
    BuyShared - Shared & Reseller Hosting / cPanel + Softaculous + CloudLinux / Pure SSD! / Free Dedicated IP Address
  • First-RootFirst-Root Member, Provider
    edited December 2018

    did someone already mentioned that the end is nigh?

    Thanked by 2eol coreflux
  • @Francisco said:

    @ersite said:
    Hi!
    My vps is down over 24 hours since yesterday. I have submitted a ticket to buyvm but no one replies me. Does anyone have the same issue? I cannot boot up my vps!! It is very bad news during the Xmas holiday.

    Do you have a ticket?

    We've been fixing up anyone that got banged up by the power outage that hit the storage rack.

    Some VM's did get hosed so we've done credits, rebuilds, etc.

    Francisco

    My ticket is #415734

  • FranciscoFrancisco Top Provider

    ersite said: My ticket is #415734

    I'll get a reply to you in the next bit :)

    FR_Michael said: did someone already mentioned that the end is nigh?

    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF

    Francisco

    Thanked by 3eol tcp6 netomx
    BuyVM - Free DirectAdmin, Softaculous, & Blesta! / Anycast Support! / Windows 2008, 2012, & 2016! / Unmetered Bandwidth!
    BuyShared - Shared & Reseller Hosting / cPanel + Softaculous + CloudLinux / Pure SSD! / Free Dedicated IP Address
  • First-RootFirst-Root Member, Provider

    Sorry buddy, someone needed to mention that. :(

    Thanked by 2eol coreflux
  • @FR_Michael said:
    did someone already mentioned that the end is nigh?

    Not that I know of.

    Thanked by 1tcp6

    ˙ɹǝuzʇǝɥ

  • deankdeank Member, Troll
    edited December 2018

    The end is not nigh because it's Frantech.

    I pity the OP though.

    Thanked by 3eol tcp6 netomx

    There are two things that make Earth spin: Money and PMS.

  • LEGACY-03.LV.BUYVM.NET is up now.

    ˙ɹǝuzʇǝɥ

  • deankdeank Member, Troll

    Fran needs to name a node after me though.

    ENDIS_NIGH_WAS_HERE.BUYVM.NET.

    There are two things that make Earth spin: Money and PMS.

  • F

    Thanked by 2tcp6 yomero

    ˙ɹǝuzʇǝɥ

  • @jvnadr said:

    ersite said: My vps is down over 24 hours since yesterday. I have submitted a ticket to buyvm but no one replies me. Does anyone have the same issue? I cannot boot up my vps!! It is very bad news during the Xmas holiday.

    Do you have backups? If yes, spin a temp vps on a service like DO or Scaleway and be online in minutes until the issue being resolved, to avoid loosing millions due to downtime.
    If you don't have backups, learn your lesson and keep backups. No infrastructure stays for ever.

    Demanding immediate response on a Sunday 3 days before Christmas for a ~$5 vps and opening a bash thread as your first and only "contribution" on LET, without even backing up your claims (e.g. issue to your own vps, not the node itself), won't give you any sympathy here.

    This is really an unhelpful post. The OP did nothing wrong, didn't slag the provider ( it's not a bash thread, learn the difference) and just asked if anyone else had same problem. He's not asking for sympathy, or advice from non BuyVM customers that isn't relevant to the OP's problem.

    He didn't say he needed to be up in 5 minutes, just needed to know wtf was going on. I've got shit to do, I don't have time to restore servers and migrate people, unless I know it will take X long or its not coming back (hence why someone posts to determine). Every one doesn't have these super basic, 5 minute switchover and back up. That's naive. There's DNS propagation, for fucks sake.

    Buyvm says they have 24/7 support. So expecting a response within 24 hours, regardless of Sunday, is again, reasonable. Regardless of price. If support doesn't answer in 24 hours, they are under water or poorly ran.

    Turns out, they did have a big problem! And despite saying he had a ticket in OP, Fran asks if he has a ticket because they're dealing with down servers by ticket. So sounds like ticket backlog.

    My question is, what does the network status say for BuyVM? It sounds like Buyvm didn't email any network status updates to the effected customers. But there should be a record of the affected servers with a timeline provided.

    People need to think more about solving OP's problems and less chastising over bullshit (provider bashing, having backups, wanting support... fuck off if you're not going to contribute).

  • @TimboJones said:
    fuck off if you're not going to contribute

    ...

    Thanked by 1tcp6

    ˙ɹǝuzʇǝɥ

  • I spent the whole day, and I found that my 500g of data was unrecoverable, damn it.

  • Better than a whole pound!

    Thanked by 2eol netomx

    Me, myself and a lowest common denominator named Bob - We all ❤️ @eol.

  • eoleol Member
    edited December 2018

    EDIT:
    500g of SSD might be more than 500g of HDD.
    Or maybe not...

    Thanked by 2tcp6 3606202

    ˙ɹǝuzʇǝɥ

  • How many inches of SQL did you lose?

    Thanked by 2eol coreflux

    Me, myself and a lowest common denominator named Bob - We all ❤️ @eol.

  • I lost 435 liters of tar.gz.

    Thanked by 2tcp6 3606202

    ˙ɹǝuzʇǝɥ

  • TimboJones said: People need to think more about solving OP's problems and less chastising over bullshit (provider bashing, having backups, wanting support... fuck off if you're not going to contribute).

    Thank you, I am going to follow your advice and will fuck off.

    (but I think that "members" with just ONE post telling about the downtime of a respected provider with a tone of bashing should also fuck off)

    TimboJones said: 5 minute switchover and back up. That's naive. There's DNS propagation, for fucks sake.

    Ah, this: somebody that runs a live website in a KVM VPS, is expected to have technical knowledge and a backup plan. So, there are plenty of services, free and paid, that can deal with DNS (cloudflare does excellent job on this field without the optimization) and an ip move will not need any propagation, it's an one minute job. And of course, there are several other methods for redundancy, failover etc.

    Sometimes I maybe am a little bitter, but I get really pissed of from people just coming to LET with a single post with also a bitter tone to ask for attention about an outage from a good provider.
    But you accusing me not being helpful (when my knowledge can help), is a little too much...

  • @jvnadr said:

    TimboJones said: People need to think more about solving OP's problems and less chastising over bullshit (provider bashing, having backups, wanting support... fuck off if you're not going to contribute).

    Thank you, I am going to follow your advice and will fuck off.

    (but I think that "members" with just ONE post telling about the downtime of a respected provider with a tone of bashing should also fuck off)

    TimboJones said: 5 minute switchover and back up. That's naive. There's DNS propagation, for fucks sake.

    Ah, this: somebody that runs a live website in a KVM VPS, is expected to have technical knowledge and a backup plan. So, there are plenty of services, free and paid, that can deal with DNS (cloudflare does excellent job on this field without the optimization) and an ip move will not need any propagation, it's an one minute job. And of course, there are several other methods for redundancy, failover etc.

    Sometimes I maybe am a little bitter, but I get really pissed of from people just coming to LET with a single post with also a bitter tone to ask for attention about an outage from a good provider.
    But you accusing me not being helpful (when my knowledge can help), is a little too much...

    OP had NO tone and reasonable request looking for info. That's my point. He waited an appropriate amount of time and then reached out.

    There was a post the other day from a guy pissed he didn't get an email after several days of downtime and didn't file a ticket or look at the network status page but demanded being told what was in the network status update already. You'd have a case with that guy bashing provider, but not this guy.

    The provider wasn't bashed and you didn't need to rush to defend BuyVM when you didn't know the story.

    Some people seem to give a lot of leeway to good guys. That doesn't mean they run a good business. It might actually mean they don't run a tight business.

    Datacenters are expected to have redundancy and power isolation so that servers are not taken out in a storm. 24 hours downtime is what you see from the likes of hiformance (now that, you can interpret as bashing). I don't think I'm incorrect that there's been more than one 24 hour+ outage in the last two years at BuyVM.

    If my providers of choice was down 24 hours without update, I would be working on moving services - permanently.

    Lastly, you're really not experienced in diverse customer applications if you think the only impact of DNS change is on the server side. For the first two applications that came to mind, a full restart of the service with downtime would be experienced (instead of limping along and immediately working again if original server came back up). For an NTP server added to the NTP pool, their IP is given to the client. I still get NTP requests years after the server dying and being withdrawn from NTP pool.

    But you'd agree, knowing when to deploy the backup plan really is based on severity of the problem and the ETA for resolution. You can't easily make that decision when provider doesn't respond or provide updates. Especially, if they are known for getting back up quickly, it's more hassle to deploy the backup prematurely.

  • FranciscoFrancisco Top Provider

    TimboJones said: My question is, what does the network status say for BuyVM? It sounds like Buyvm didn't email any network status updates to the effected customers. But there should be a record of the affected servers with a timeline provided.

    We emailed people an update on everything yesterday and informed anyone still having issues with their storage vm's to please ticket and i'll look after them.

    I've been working through queue helping people with FSCK's and things like that.

    There's been no tickets sitting for 24 hours without a reply. The only ticket with any amount of time on it is me assisting someone with a project, everything else is being handled quickly :)

    As for the OP's ticket, they originally ticketed at 7AM and I got my 1st reply on them around lunch time as I was handling some other tickets.

    The OP will be fixed and given a credit that I hope he feels is fair.

    It's a lame ass festivus, but we'll figure it out.

    Francisco

    Thanked by 3TimboJones t0m loyd
    BuyVM - Free DirectAdmin, Softaculous, & Blesta! / Anycast Support! / Windows 2008, 2012, & 2016! / Unmetered Bandwidth!
    BuyShared - Shared & Reseller Hosting / cPanel + Softaculous + CloudLinux / Pure SSD! / Free Dedicated IP Address
  • PieHasBeenEatenPieHasBeenEaten Member, Moderator

    Well this gives the whole story now thanks! Sounds like someone was impatient.

  • FranciscoFrancisco Top Provider
    edited December 2018

    @PieNotEvenEaten said:
    Well this gives the whole story now thanks! Sounds like someone was impatient.

    No, I was simply asleep when the initial ticket came in.

    The OP's not at fault at all, this is squarely on me for not addressing the 2nd power feed back in August.

    Truth be told I came back from Vegas and was in the midst of finalizing things but ran into some annoying medical issues so i've been preoccupied with that for the entire year (still dealing with it now). It simply slipped my mind and bit me.

    Francisco

    Thanked by 1zed
    BuyVM - Free DirectAdmin, Softaculous, & Blesta! / Anycast Support! / Windows 2008, 2012, & 2016! / Unmetered Bandwidth!
    BuyShared - Shared & Reseller Hosting / cPanel + Softaculous + CloudLinux / Pure SSD! / Free Dedicated IP Address
  • PieHasBeenEatenPieHasBeenEaten Member, Moderator

    @Francisco You always own up to your mistakes and came up with solutions to those mistakes. But there has to be patience also from both sides of the isle. Well, anyways hope you feel better.

  • deankdeank Member, Troll

    Sleeping has become a sin in the days of the internet.

    Repent, Fran. Name a node after me. Then you shall be forgiven, at least by me.

    There are two things that make Earth spin: Money and PMS.

  • FranciscoFrancisco Top Provider

    deank said: Repent, Fran. Name a node after me. Then you shall be forgiven, at least by me.

    Will do, when i'm in Vegas in the spring i'll put a "deank" label on one of the nodes, just for you.

    Francisco

    Thanked by 3deank edit2 eol
    BuyVM - Free DirectAdmin, Softaculous, & Blesta! / Anycast Support! / Windows 2008, 2012, & 2016! / Unmetered Bandwidth!
    BuyShared - Shared & Reseller Hosting / cPanel + Softaculous + CloudLinux / Pure SSD! / Free Dedicated IP Address
  • @Francisco Happy festivus! Do you want me to bring you a pole?

  • FranciscoFrancisco Top Provider

    @doghouch said:
    @Francisco Happy festivus! Do you want me to bring you a pole?

    I don't swing that way, sorry.

    Francisco

    BuyVM - Free DirectAdmin, Softaculous, & Blesta! / Anycast Support! / Windows 2008, 2012, & 2016! / Unmetered Bandwidth!
    BuyShared - Shared & Reseller Hosting / cPanel + Softaculous + CloudLinux / Pure SSD! / Free Dedicated IP Address
  • @Francisco said:

    @doghouch said:
    @Francisco Happy festivus! Do you want me to bring you a pole?

    I don't swing that way, sorry.

    Francisco

    :-(

  • Still having issues with the block storage. Sounds more than a FS broken but like an underlying device issue. My kernel says it got some I/O errors when accessing the device. So still waiting for Frans to solve the issue (if he's not asleep yet).

  • JohnMiller92JohnMiller92 Member
    edited December 2018

    Francisco said: Truth be told I came back from Vegas and was in the midst of finalizing things but ran into some annoying medical issues so i've been preoccupied with that for the entire year (still dealing with it now). It simply slipped my mind and bit me.

    Hope you get better!

  • FranciscoFrancisco Top Provider

    @msg7086 said:
    Still having issues with the block storage. Sounds more than a FS broken but like an underlying device issue. My kernel says it got some I/O errors when accessing the device. So still waiting for Frans to solve the issue (if he's not asleep yet).

    I'll get a reply in a few.

    We had a interesting case that I bet is the same as yours related to QCOW images. The image simply...didn't know how big it was. A quick fix, but it would make it so the user would see the wrong size inside of their VM.

    Francisco

    BuyVM - Free DirectAdmin, Softaculous, & Blesta! / Anycast Support! / Windows 2008, 2012, & 2016! / Unmetered Bandwidth!
    BuyShared - Shared & Reseller Hosting / cPanel + Softaculous + CloudLinux / Pure SSD! / Free Dedicated IP Address
  • @Francisco was transparent about the situation and was on top of the issue with my block storage. I feel like I should be paying him a lot more for the service I receive.

  • tcp6tcp6 Member
    edited December 2018

    @Xenos said:
    @Francisco I feel like I should be paying him a lot more

    JUST DO IT!

  • Daniel15Daniel15 Member
    edited December 2018

    @dajiba said:
    I spent the whole day, and I found that my 500g of data was unrecoverable, damn it.

    Restore from backups? You do have backups, right?

    My VPS was fine after the power outage, but my slab was corrupted to the point where it won't even mount any more. No worries, I do daily backups so I just restored from backup onto the VPS, and will wipe the slab, reformat it, and move everything back once I've got some free time.

    If you don't have backups, maybe this is a wakeup call. :)

    Thanked by 2tcp6 eol
  • @Daniel15 said:
    If you don't have backups, maybe this is a wakeup call. :)

    Quoted for truth.

    Thanked by 3eol gestiondbi t0m
  • deankdeank Member, Troll

    Backups are for pussies. Real men take real risks and shed manly tears later.

    Be a man and don't make backups.

    Thanked by 4Shazan eol Etonix t0m

    There are two things that make Earth spin: Money and PMS.

  • @deank said:
    Backups are for pussies. Real men take real risks and shed manly tears later.

    Be a man and don't make backups.

    Condoms are for pussies. Real men take real risks and shed manly tears later.

    Be a (dead) man and gamble with your life.

  • Thanks all. The issue has been corrected! My data does not lose but it takes time to recover. I also got the 3-month credits. I use this VPS to backup other servers. I will consider doing a local backup on my computer for this VPS once a week.

  • Irony is, This one was my BACKUP server :neutral:
    And, It had backup of few of my sites that I terminated last week. I was going go live with this backups on new server after this holiday. :disappointed: Yesterday, I was going to download those backup on my local machine, and this had to happen.

    I guess I need to keep backup of my backup.

    I know I won't get any sympathy here, But can anybody suggest a hosting provider where I can store backup of my backup?

    Thanked by 1kkrajk
  • @Ataurbd said:
    I know I won't get any sympathy here, But can anybody suggest a hosting provider where I can store backup of my backup?

    Get a cheap storage VPS, or use something like Backblaze B2.
    https://www.lowendstorage.win/ is a good reference. Hetzner have some good storage boxes too, particularly if you want to store a lot of data.

  • @Ataurbd said:
    I guess I need to keep backup of my backup.

    Ideally you have a backup of the backup of the backup.

    ˙ɹǝuzʇǝɥ

  • tcp6 said: Condoms are for pussies

    Yeah, use them the way God intented!

    Thanked by 1eol
  • Mine didn't survive since the underlying device is gone due to some software bugs. I was offered compensation and I'm gonna move on and start recovering data from backups. Lost maybe a few days of data since last snapshot, but nothing was critical.

    Also I'm gonna get a second volume so I can do a RAID-1 over it for extra safety.

  • FranciscoFrancisco Top Provider
    edited December 2018

    The main issue we've learned is QCOW2 is just a mess when it comes to larger volumes and any sort of repairs to it.

    We've been swapping people to RAW's and have actually converted a few people today from QCOW to RAW as well. It takes a bit of time, but in the off chance there's ever an issue, there's no funny business.

    As I mentioned, the power changes are due this month if I can help it, start of January worst case, so that rules that out of the question.

    Francisco

    Thanked by 1msg7086
    BuyVM - Free DirectAdmin, Softaculous, & Blesta! / Anycast Support! / Windows 2008, 2012, & 2016! / Unmetered Bandwidth!
    BuyShared - Shared & Reseller Hosting / cPanel + Softaculous + CloudLinux / Pure SSD! / Free Dedicated IP Address
  • @tcp6 said:
    Condoms are for pussies.

    You want a condom during penetration tests to avoid SQL injection.

    ˙ɹǝuzʇǝɥ

Sign In or Register to comment.