Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Another Intel exploit which allows VPS shutdown
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Another Intel exploit which allows VPS shutdown

stefemanstefeman Member
edited November 2019 in General

From DigitalOcean

"Hi there,

Intel released a statement regarding Machine Check Error Avoidance on Page Size Change. We have rolled out mitigation efforts across our entire fleet and are writing to let you know that our platform is secure and your Droplets and data will not be impacted and you do not need to take any action.

While customers updating to the latest kernels from OS providers may see that the status is vulnerable, we want to assure you that your Droplets are safe because we’ve implemented mitigation at the hypervisor level.

Machine Check Error Avoidance on Page Size Change is a significant security issue. If an attacker were able to create a Droplet colocated with another user’s Droplet, they could exploit the vulnerability, giving them the ability to shutdown that Droplet within a few minutes. We acted quickly to roll out our mitigations and ensure our users were not vulnerable to any attacks.

You may have also heard about the TSX Asynchronous Abort (TAA) vulnerability announced by Intel. This vulnerability does not impact our platform.

The security of our platform and our users’ data is our top priority, and we’re taking every measure to ensure our customers remain secure, including keeping you informed of updates about vulnerabilities that may impact your account. For more information, you can read Intel’s deep dive.

Thanks,
Team DigitalOcean"

Comments

  • Intel becomes new Flash... Unfixable security flaws just pilling in at rates that troubles even Adobe security engineers :-/

  • Ouch.
    And that's where small dedi (atoms...) shine, even though you can get better performance at a lower cost with a VPS.

    Some providers are already using AMD though, so far they seem to do better on the security front.

  • @jsg what's your take on this?

  • jsgjsg Member, Resident Benchmarker
    edited November 2019

    @poisson said:
    @jsg what's your take on this?

    Brutally summarizing my take is that this is just the clusterf_ck of the month (or even the week) (TM) that will soon be followed by the next horror story.

    I wouldn't be too concerned about this one although it's a serious problem. What I'm way more concerned about is the reason for this clusterf_ck series. This paragraph provides a strong hint:

    intel said:
    After the paging structure modification, but before software invalidates any ITLB entries for the linear address, code fetch happens again on the same linear address.

    It confirms something that IMO was to be presumed since quite some time now: poor engineering. Note that that does not mean that intel has lousy engineers. I guess it's more like the situation in software. There's almost everywhere almost always some product or other manager and up to top management who push the techies hard. The techies usually ask for sufficient time to design and implement things properly as well as to clean up well known viper pits, but they don't get that time and are instead pushed for new features and more performance.

    Concretely the quote shows (yet again) lack of proper fencing and atomicity as well as way too complex and not really thought through internal mechanics. Those are things engineers are very, very unhappy with and those are things some clueless non-tech manager doesn't care about. They care about features, a bit more performance, a bit lower power consumption. "It does (translation: seems to) work" is good enough.

    But while I'm confident that AMD is clearly a better choice there is a really monstrous "but": Conformance/Compatibility. Short version: If the underdog wants to sell his products they must be functionally compatible with the products of the big fat monster dog. And yes, this strongly limits AMDs option, up to a point where they must implement poorly engineered crooks because customers expect AMD processors to function (almost) identical to intels processors.

    OS, driver, etc developers make an awful lot of efforts to build to "x86 specs" which practically means to intels specs and they certainly won't do that all over again just because some underdog wants to do some things properly - which to those developers almost always translates to "differently" which translates to extra work and lots of it. So, AMD processors must - and at quite a deep level - feel/look like intel processors - incl. the quirks.

    Purely practically speaking re the current vulnerability I frankly do not trust DO's statement. What I do trust is that DO is big enough to have one or a few guys really understanding the problem. I also think that DO actually does have some mitigations - but only to a limited degree.

    You see, intel even says it albeit embedded in a lot of lingo: If you allow anyone (clients) to have their own low level code (OS, drivers, ...) then it's all doors open. Some brutally boiled down it means VPS hosting has become all but lottery.
    Also note that pretty much all processors commonly used in VPS hosting are in the bad list.

    Also note that this thing is not yet another one in the line of Spectre, Meltdown, etc. It's a different beast - and I expect quite a few more vulnerabilities to follow. So, just disabling SMT/HT does not protect against this vulnerability.

    Edit: Made the last paragraph more clear

  • Good posting @jsg! Because of all those intel vulnerabilities i got rid of all vps and only use dedicated servers / raspberry pis now. A little bit more expensive, lesser locations, but feeling more secure.

  • HostEONSHostEONS Member, Patron Provider

    Most of our VPS nodes are based on AMD but recently we got few Intel based VPS nodes, but it seems a bad decision to get Intel based servers as new Intel vulnerabilities keeps popping up every few days

  • jsgjsg Member, Resident Benchmarker

    @HostEONS said:
    Most of our VPS nodes are based on AMD but recently we got few Intel based VPS nodes, but it seems a bad decision to get Intel based servers as new Intel vulnerabilities keeps popping up every few days

    Yes, but I think it would be a mistake to assume AMD == safe.

    AMD indeed seems to have some things better than intel and from what I see AMD Zen is a much better choice but keep in mind that AMD had to stay highly compatible. Also keep in mind that some problems are more to do with both the x86 architecture and its evolution and backward compatibility.

    So, yes, buy AMD - but stay vigilant anyway.

  • @jsg Thanks man! I am so glad my I got my Ryzen potassium server from @seriesn and I will probably start dumping my Intel boxes save those that are storing non mission critical stuff (e.g. long term storage boxes).

    Thanked by 1seriesn
  • jsgjsg Member, Resident Benchmarker
    edited November 2019

    @poisson said:
    @jsg Thanks man! I am so glad my I got my Ryzen potassium server from @seriesn and I will probably start dumping my Intel boxes save those that are storing non mission critical stuff (e.g. long term storage boxes).

    Yes, provided that you pre(!)-encrypt your data before pushing it to a storage box that sounds like a sensible approach.

    Oh and, I'm also glad that I got one them really great NexusBytes Ryzen based VPS. Although frankly it is so insanely fast compared to the usual Xeon 26xx v2 and even v3 boxes that I don't have a load to really use its performance to any significant degree, haha. But large (as in lots of requests) sites with a high DB load will fly.

    Thanked by 1seriesn
  • seriesnseriesn Member
    edited November 2019

    To be honest, be it intel, be it AMD, be it any software, nothing is perfect since everything is man-made. As @jsg mentioned, stay vigilant.

    Thank you for the kind words @poisson @jsg :)

  • datanoisedatanoise Member
    edited November 2019

    Great writing as usual @jsg

    The problem we face (as cheap asses using dirt cheap VMs) is that most Low End providers don't have a clue about how those exploits really work (or even how CPUs work): they know how to select the right hardware, install the right software and manage the nodes, that's about it. Even doing their best, their stuff can't be secure, until the patches become widely available (and even then, YMMV).

    You are probably right to mention that DO is big enough to have at least a guy having a pretty good understanding of what's going on, and with lot of cash invested they have to do their best to mitigate these issues and have the funds to focus on it, at least a bit. Same could be said for all the big players. The small players of the low end market? Well... the situation is far worse than lottery imo. Unless this lottery has a very small percentage of winners.

    Profitability is what matters when margins are low and you can't necessarily disable HT or allow a patch to limit too severely your ability to oversell - ignore what's going on and don't communicate with your clients can be the way many low end providers handle these issues.

  • datanoise said: The problem we face (as cheap asses using dirt cheap VMs) is that most Low End providers don't have a clue about how those exploits really work (or even how CPUs work): they know how to select the right hardware, install the right software and manage the nodes, that's about it. Even doing their best, their stuff can't be secure, until the patches become widely available (and even then, YMMV).

    We raise our hand, scream #yolo and go to sleep every night, hoping to wake up to nothing broken.

    Thanked by 1datanoise
  • jsgjsg Member, Resident Benchmarker

    @datanoise said:
    Great writing as usual @jsg

    The problem we face (as cheap asses using dirt cheap VMs) is that most Low End providers don't have a clue about how those exploits really work (or even how CPUs work): they know how to select the right hardware, install the right software and manage the nodes, that's about it. Even doing their best, their stuff can't be secure, until the patches become widely available (and even then, YMMV).

    You are probably right to mention that DO is big enough to have at least a guy having a pretty good understanding of what's going on, and with lot of cash invested they have to do their best to mitigate these issues. Same could be said for all the big players. The small players of the low end market? Well... the situation is far worse than lottery imo. Unless this lottery has a very small percentage of winners.

    I'm a bit less pessimistic than you because there is more to actually exploiting that vulnerability than just running am evil script (if one exists anyway) plus an attacker also f_cks up his VPS which may be irrelevant for larger players but will make think scriptkiddies twice.

    As for large and small providers, keep in mind that while large provider have at least one or a few guys actually understanding the vulnerability large companies are also the one who highly likely lie because they can afford a PR department and good lawyers too.

    My current impression is that most players wait for a miracle patch from the OS and hypervisor people. I don't hold my breath but I presume that at least some limited mitigation will become available.

    As I said, the real problem is that almost everybody commits the same sins incl. the (no matter whether foss or not) OS and hypervisor people,plus the vulnerability is very very ugly especially for any kind of VPS hosting. Reason: About the only idea I see so far (other than simply closing the shop or not caring at all) would be to allow only some prepared (and checked) guest OSs but (a) how to enforce and control that? and (b) that only shifts the problem rather than solving it for diverse reasons, an important of which is how to keep people from installing their own OS, kernel, drivers - and even if you could do that that would just add one more (rather thin for professionals) layer evil guys had to cross.

    All in all I expect this vulnerability to be way worse than the Spectre family because (a) it's only the first one, others will follow, and (b) there is no simple (albeit expensive like disabling SMT/HT) solution.

    Thanked by 1datanoise
  • NeoonNeoon Community Contributor, Veteran

    Planes are magic, okay.

Sign In or Register to comment.