Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


VULTR Seattle Node Failure - Page 2
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

VULTR Seattle Node Failure

2

Comments

  • ViridWebViridWeb Member, Host Rep

    @Pwner said:

    @ViridWeb said:
    Last, I don't know what language you use on your support ticket but the host should not threaten their clients to close their account without any reason.

    If you read the OP, you'd see the following:

    @pixelpadre said: I sent a support request to close my account. I indicated my dismay and disgust with their company. I quickly got a response informing me that they would close my account without any apologies or explanations that any respectable company would have offered.

    OP requested for their account to be closed, Vultr support didn't make any threats, they were following through with OP's request.

    If you read carefully then you will find I already have acknowledged my mistake

    @ViridWeb said:

    @sanvit said:

    @ViridWeb said: host should not threaten their clients to close their account without any reason.

    OP never mentioned that they threatened to close his/her account?

    Anyway, you should always have backup. period.

    Yep you are right. I just double checked.

  • HostSlickHostSlick Member, Patron Provider
    edited June 2020

    @pbx said:
    Do they officially even use RAID now or is still "maybe yes maybe no"?

    Theres a low % of chance that data loss can happen with RAID1 or even 10 aswell.
    For example, We had a node with RAID10 and bad luck last year too. Two drives failed at the same time of whom one of each span.

    After we replaced the failed drive, hardware raid was doing its sync when it suddenly marked another one as failed.

    There was no sign before that the other one would fail.

    Both SSD Drives where about 3 months old.

    Always keep your backups. RAID is no backup but redundancy to lower the risk of loss and such events like data loss to occur.

    Just saying :wink:

  • @pbx said: Do they officially even use RAID now or is still "maybe yes maybe no"?

    I asked this is what they told me:

  • ViridWebViridWeb Member, Host Rep

    @jahrinc said:

    @pbx said: Do they officially even use RAID now or is still "maybe yes maybe no"?

    I asked this is what they told me:

    I think the guy was talking about instances/VPS not the node itself. Or maybe he was just trying to avoid the question

  • @ViridWeb said: I think the guy was talking about instances/VPS not the node itself. Or maybe he was just trying to avoid the question

    Welp... :

  • @AC_Fan said:
    Everything dies, eventually. To pretend otherwise is naive. Keep backups from now on and go on with your life.

    Since it is a CloudVPS and the OP does not have access to the physical node to shutdown and image the whole drive, how do you backup?
    rsync from the root? tar from the root and copy the tar file?
    I agree this thread turned sour, but a little education on how to backup and re-install on another VPS would be appreciated.

  • If you need redundant storage, Vultr's block storage is SAN / Replicated.

  • tetechtetech Member

    @techwiz44 said:

    @AC_Fan said:
    Everything dies, eventually. To pretend otherwise is naive. Keep backups from now on and go on with your life.

    Since it is a CloudVPS and the OP does not have access to the physical node to shutdown and image the whole drive, how do you backup?
    rsync from the root? tar from the root and copy the tar file?
    I agree this thread turned sour, but a little education on how to backup and re-install on another VPS would be appreciated.

    Vultr control panel. Backups tab -> Enable backups.

    But I don't think the complaint here was about loss of data anyway.

  • Tony40Tony40 Member
    edited June 2020

    Shit always happen, you should always backup to a remote location or download... personally, I ways download a copy of the full backup...In my pc I have a external hard drive plug-in that auto backup my pc files, so is in three location.

  • Better ask your mother for a refund if you suffer erectile dysfunction one day.

    Thanked by 1CConner
  • pbxpbx Member

    @ViridWeb said: trying to avoid the question

    They've been pretty good at that over the years :wink:

  • @techwiz44 said:

    @AC_Fan said:
    Everything dies, eventually. To pretend otherwise is naive. Keep backups from now on and go on with your life.

    Since it is a CloudVPS and the OP does not have access to the physical node to shutdown and image the whole drive, how do you backup?
    rsync from the root? tar from the root and copy the tar file?
    I agree this thread turned sour, but a little education on how to backup and re-install on another VPS would be appreciated.

    I've not tried but apparently the beta version of Urbackup can now take a full image of running systems (Depending on Filesystem) using the dattobd kernel module

    https://forums.urbackup.org/t/server-2-5-9-beta-updated-x3/8865

    I believe VEEAM can also do similar via it's kernel module.

  • Anyone here know if DO or Vultr use local disk for their root drive instance or use network block storage/CEPH?

  • @danninov said:
    Anyone here know if DO or Vultr use local disk for their root drive instance or use network block storage/CEPH?

    DO use local RAID storage. Vultr use local but have avoided answering if they use any RAID.

    Thanked by 1pbx
  • @Brend4n said:

    @danninov said:
    Anyone here know if DO or Vultr use local disk for their root drive instance or use network block storage/CEPH?

    DO use local RAID storage. Vultr use local but have avoided answering if they use any RAID.

    Ah okay, I thought they like EC2 AWS that using block storage for root drive. No protection then from failure node, quite high risk

  • @poisson said:
    Better ask your mother for a refund if you suffer erectile dysfunction one day.

    I don't know if erectile dysfunction analogy is correct, it would be more like the mother gave him crabs and offered a free handy but didn't apologize for the crabs.

    Also, jeepers, man. Did he start something personal with you outside this thread that you need to go after his mom? (It would be criminal if tomorrow was mother's day instead of Father's Day).

  • @Brend4n said:

    @danninov said:
    Anyone here know if DO or Vultr use local disk for their root drive instance or use network block storage/CEPH?

    DO use local RAID storage. Vultr use local but have avoided answering if they use any RAID.

    I don't see it mentioned for Compute or in their SLA. So they're not giving the false impression.

    When the provider isn't using RAID10, server/data loss is eventual and so automatic backups are mandatory. And even if they are, they can still fuck it up.

  • @danninov said:

    @Brend4n said:

    @danninov said:
    Anyone here know if DO or Vultr use local disk for their root drive instance or use network block storage/CEPH?

    DO use local RAID storage. Vultr use local but have avoided answering if they use any RAID.

    Ah okay, I thought they like EC2 AWS that using block storage for root drive. No protection then from failure node, quite high risk

    If you want HA you have to pay the price (not lowend definitely) and in most cases suffer the performance hit that comes with it. In most cases disks are whats most likely to fail on you and if theres local RAID you’d be looking at potentially going years without issues. But this is why backups are important...

  • @WSCallum said:

    @danninov said:

    @Brend4n said:

    @danninov said:
    Anyone here know if DO or Vultr use local disk for their root drive instance or use network block storage/CEPH?

    DO use local RAID storage. Vultr use local but have avoided answering if they use any RAID.

    Ah okay, I thought they like EC2 AWS that using block storage for root drive. No protection then from failure node, quite high risk

    If you want HA you have to pay the price (not lowend definitely) and in most cases suffer the performance hit that comes with it. In most cases disks are whats most likely to fail on you and if theres local RAID you’d be looking at potentially going years without issues. But this is why backups are important...

    I mean, providers that have network/redundant/CEPH storage are pretty comparable in pricing these days... UpCloud and Heficed come to mind. Heficed use CEPH with triple replication, and UpCloud their own inhouse I believe.

  • SplitIceSplitIce Member, Host Rep
    edited June 2020

    Mate you have been at this for years and you still havent learnt to take backups? How about instead of trying to place blame on them take some responsibility. This was bad luck for them which became a disaster for you due to missmanagement. Perhaps there is a lesson to be had here?

    Snapshots are free with Vultr, so there is really no reason not to use them.

    Further if this was a regular occurrence with them they wouldnt be as big as they are now. I'd say it's not based on my ~35 VMs, many of which are 4y+ old. I'm yet to see a single full Node failure that they didn't recover (one they did) or in recent years any serious issues. Comparing that to many of the hosts around here... yeah.

    Anyway Im not saying they are perfect (I've taken issue with some of their policies over the years) but they are more than adequate and definitely in the top tier for the price bracket.

    Thanked by 2hohl raindog308
  • @SplitIce said:
    Mate you have been at this for years and you still havent learnt to take backups? How about instead of trying to place blame on them take some responsibility. This was bad luck for them which became a disaster for you due to missmanagement. Perhaps there is a lesson to be had here?

    Get off your high horse, he was testing it. You didn't even read the complaint.

    Snapshots are free with Vultr, so there is really no reason not to use them.

    That's the useful takeaway advice that's key to explaining why this provider doesn't use RAID.

    However, one hour of my time is more than any month of service costs, so even having to restore from backups is a money and time loser not to mention potential loss of millions (hehe). It's worse when you're a one man shop and also want a life.

    Further if this was a regular occurrence with them they wouldnt be as big as they are now. I'd say it's not based on my ~35 VMs, many of which are 4y+ old. I'm yet to see a single full Node failure that they didn't recover (one they did) or in recent years any serious issues. Comparing that to many of the hosts around here... yeah.

    Anyway Im not saying they are perfect (I've taken issue with some of their policies over the years) but they are more than adequate and definitely in the top tier for the price bracket.

    You still fail to see his point? You pretty much made it. You've had 4+ years with 35+ servers and minimal problems (technically no data loss?). Now, go try another service and if you have data loss in first month, see if you'd hold them in same light as you do with Vultr.

    It's bad luck odds. The same shit happens with Seagate and WD hard drives. Some people buy one brand for years with no problems and then get early failure on just one of the other brand, and now that brand is garbage for life. It's only the people who deal in quantity that can see the bigger picture.

  • SplitIceSplitIce Member, Host Rep

    @TimboJones You are arguing against having your own backups and arguing for making rash decisions. I'm going to assume you are trolling.

  • TimboJonesTimboJones Member
    edited June 2020

    @SplitIce said:
    @TimboJones You are arguing against having your own backups and arguing for making rash decisions. I'm going to assume you are trolling.

    No, I'm saying he has a valid complaint and being told that he should have backups does fuck all to prevent the inconvenience (the complaint) of that data loss.

    The learned lesson here is to pick a provider who uses RAID 10. He might have just assumed it because of the SLA promise, or that every single other datacenter he's ever used had RAID and expected it to be common place. He didn't go with a properly redundant setup for most reliability and therefore it's inevitable that there's data loss. This is key lesson learned here.

    Let's say you bought a new car. You're expecting a nice, reliable vehicle for the first year. But the fucker breaks down on the way to an important meeting. Being told, "well, don't complain, YOU take it to the repair shop, YOU go without while it's being repaired, and YOU pick it up again afterwards". Just because it was repaired under warranty doesn't get you the time and inconvenience back.

    Most people are ok with restoring from backups one every four years for non preventable errors. Not in first 6 months or when first trying a new service. I lost multiple VM's from Oracle in first few months and they are dead to me for life. I lost no money or irreplaceable data, but my experience in that brief time will forever be remembered.

    People want RAID setups to reduce the likelihood of downtime. Having backups for irreplaceable data is a requirement in any circumstance and doesn't do anything for uninterrupted operation or the root complaint by OP.

  • @SplitIce said:
    @TimboJones You are arguing against having your own backups and arguing for making rash decisions. I'm going to assume you are trolling.

    I dont think anybody is against taking their own backups. More so, a production level provider should have redundancy in place to attempt to mitigate these cases. Of course hardware failures happen; however, if there was RAID or network/replicated storage, restoring from a backup would be worst case. I'd prefer to have as much disaster recovery in place as possible before manual intervention.

  • fpmagicfpmagic Member
    edited June 2020

    It is a little surprising that there is no redundancy against a single drive failure for the whole host node, that said the backup message is clear.

    People want RAID setups to reduce the likelihood of downtime.

    RAID or software emulation, or some fancy ZFS style, or CEPH or, AWS EBS style, the implementation detail aside, some level of redundancy is just considered normal nowadays, but once this doesn't mitagate the need for backups.

  • @TimboJones said:
    No, I'm saying he has a valid complaint and being told that he should have backups does fuck all to prevent the inconvenience (the complaint) of that data loss.

    You literally said this in your previous reply:

    @TimboJones said:
    It's bad luck odds.

    We cannot expect not to be inconvenienced totally, right? Furthermore, in that same reply about bad luck odds, you said this:

    @TimboJones said:
    It's only the people who deal in quantity that can see the bigger picture.

    Vultr is a huge company with more nodes than we can imagine. One node failing in the bigger picture of things considering the quantity of nodes across time is, in your own words, bad luck odds. Sure, you cannot get your time back, but we all have been inconvenienced by bad luck odds at several points in our lives, and we accept it as normal.

    Personally, I don't think this case reflects badly on Vultr. If another 3-4 node failures show up, then I accept the evidence that there is something systematically wrong with Vultr. As of now, the evidence points to bad luck and that's just a fact of life.

  • @poisson said:

    @TimboJones said:
    No, I'm saying he has a valid complaint and being told that he should have backups does fuck all to prevent the inconvenience (the complaint) of that data loss.

    You literally said this in your previous reply:

    @TimboJones said:
    It's bad luck odds.

    Yes. One guy goes 4+ years on 35+ servers without data loss and another goes 6 months. It wasn't targeted or intentional, so I'm calling that bad luck, not malicious or incompetence.

    We cannot expect not to be inconvenienced totally, right?

    Correct. Data loss is inevitable, but how often it occurs can statistically be reduced and would be with RAID 10. The smart decision is to go with the least chance of inconvenience. Turns out, he didn't go for the right product. That is THE most important takeaway as that was his number one priority and what he cared about most.

    Furthermore, in that same reply about bad luck odds, you said this:

    @TimboJones said:
    It's only the people who deal in quantity that can see the bigger picture.

    Vultr is a huge company with more nodes than we can imagine. One node failing in the bigger picture of things considering the quantity of nodes across time is, in your own words, bad luck odds. Sure, you cannot get your time back, but we all have been inconvenienced by bad luck odds at several points in our lives, and we accept it as normal.

    Yes, from Vultr point of view. From the OP, he sees it closer to 100% failure and that they treat it as everyday occurrence rather than the "act of God" rare occurrence he's expecting. If you get 4 years reliable service before running into the first failure, you're going to be a lot more forgiving than someone who failed on first server. They don't have crystal ball to know if it was rare or red flag of incompetence.

    Personally, I don't think this case reflects badly on Vultr. If another 3-4 node failures show up, then I accept the evidence that there is something systematically wrong with Vultr. As of now, the evidence points to bad luck and that's just a fact of life.

    Looking at their site, they're not misleading on the lack of storage redundancy, so you get what you pay for. I'm more saying, that hearing this guy complain, he needs to hear he bought the wrong product.

    Thanked by 1poisson
  • nbnnbn Member

    I've had great luck with the high frequency VPS from vultr, and they aren't much of an upcharge. You get a beast ryzen cpu and I haven't had any crashes. Shit happens, but vultr now also has snapshots making backups really easy to create...

  • @nbn said:
    I've had great luck with the high frequency VPS from vultr, and they aren't much of an upcharge. You get a beast ryzen cpu and I haven't had any crashes. Shit happens, but vultr now also has snapshots making backups really easy to create...

    Vultr uses Ryzen now? :open_mouth:

  • @Brend4n said:

    @nbn said:
    I've had great luck with the high frequency VPS from vultr, and they aren't much of an upcharge. You get a beast ryzen cpu and I haven't had any crashes.

    Vultr uses Ryzen now? :open_mouth:

    I use Vultr HF VPS and it is not Ryzen:

    Vendor ID: GenuineIntel Model name: Intel Core Processor (Skylake, IBRS) CPU MHz: 3792.048

Sign In or Register to comment.