New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Anyone using Ryzen CPUs for Virtualization?
randvegeta
Member, Host Rep
in General
Haven't touched any AMD CPUs in over a decade.
I've heard good things about these Ryzen 7s. Decent performance, good price, and apparently not plagued with the recent Intel bug.
Any one using these for VPS nodes?
Comments
Only PART of that bug is Intel-specific. It's still a buggy turd. Nodes? Not sure, but @fLoo likely has a few ideas by now..
I'm surprised no one using these CPUs. I suppose I'll just buy a few and report back on performance.
Big issue is the lack of IPMI on any of the Ryzen boards.
Francisco
This. And KVM isnt optimized for AMD, Since 98% of the server market is intel cpus
Yeah but 99% of Intel CPU have that nasty little bug
.
Yes very unfortunate. Luckily I have a KVM switch and PXE boot server which should help. But I'm considering building a small HA cluster anyway, which should mitigate the urgency for needing kvm access.
I do have and run on a Ryzen. Positive experience so far. But ...
Zen procs are not immune. They are only partially immune. I'd assume, though, that amd is going to be quicker than intel to offer a modern, powerful and S/M immune (zen) proc, namely with their anyway upcoming new ryzen version (It's way easier to adapt a still somewhat floating design than one that is out there in billions of processors).
You do not want ryzen for server boards but "Epyc" (the high end/server version).
Running it locally in VMware and Virtualbox with no problems. Haven't tried with a full virtualisation however.
Im all for it to try ryzen with KVM or Xen, we can Perhaps test it out on a hetzner box?
Don’t be cheap and take Epyc.
Ryzen is not made for servers. It’s almost like putting I5 in, but with one less bug.
I'll just throw away all my E5s while I'm at it!
Actually it's not so much about price that puts me off Epyc. It's the complete lack of availability in HK. Would need to order it all in and that would take weeks. Need some boxes NOW, and it's either AMD desktop stuff, or Intel's bug infested server CPUs.
This is for a budget deployment high availability cluster. Sad to lose the IPMI that I have on every other box, but as mentioned above, over-provisioning a few nodes should mitigate the urgency of a failed box. The savings on the initial costs more than make up for over-provisioning a few boxes.
What's the big down side? I've got a number of budget nodes operating in LT running on i3, i5 and i7s, and they have all been running remarkably well. They are old models uses by client's who wanted to save as much as possible on the monthly rental and did not care about remote management tools (so not my first choice to buy them). But they have been surprisingly solid little runners. Never had any issue with any of them (other than the odd disk failure). Would Ryzens be any different?
@randvegeta
What you could so is to buy some Ryzen desktop motherboards for purposes of testing the zen architecture with KVM, VM, whatever you might be interested in - plus - calmly wait for the EPYC mainboards or servers to be delivered once your testing tells you to go that route.
Two remarks: While Ryzen is certainly not the most professional and high performance way to go about things, those testing Ryzen systems wouldn't be wasted. You could use them for office purposes (e.g. NOC) or even for some VPS machines; after that's 8 3+ GHz cores and 64 GB memory. We all that there are far worse nodes than that out there. Of course, the reasonable condition (lacking ipmi) would be to have them within physical access (or expensive kvms...).
Secondly: While I personally have an Asus board (and can recommend it) you might want to have a closer look at MSI as these seem to be the "official board partners" for zen.
Xeon E3 is identical to a i5/i7, just with ECC RAM support (and nowadays, different socket too). Intel segregates the market this way because it makes them more $$$.
Arguably, Xeon E3 isn't really Intel's true server platform, but they consider it good enough to market it as a server/workstation chip.
Ryzen, on the other hand, supports ECC (dependent on motherboard). Epyc's clockspeeds, core count and I/O are somewhat more ideal for a typical (large) server. I believe there's also some extra validation which goes into Epyc (+motherboards), as well as Intel's true server platform (Xeon E5 or Xeon [precious metal here]), so there's probably some benefit to using those over desktop chips.
As such, I'd argue that using a Ryzen in a server is about equivalent to using a Xeon E3 in one. Perhaps 'cheap', but should be fine for most.
The purpose is to build a small cluster for budget VMs. I am very happy to migrate to some proper EPYC CPUs if the platforms are compatible/easy to migrate between. After all, this is for a HA cluster, so if I could add EPYC cpus to the cluster, it should be pretty easy.
We have hundreds of machines, and plenty of old ones that are completely useless for anything 'production'. We COULD use the Ryzen's in the NOC/Office, but they are not actually needed since we are already using a bunch of old Xeons and the odd i5/i7 already.
Unless completely useless, the Ryzen's will be destined for running VMs over Xen or KVM. One way or another, whatever machines we build out now, will become a VPS node.
Due to the complete lack of availability of AMD Server CPUs, the only choices now are Ryzen or Xeon E3, E5, Intel Scalable. I'm not sure buying more of the bug riddled CPUs are worth it at the moment...
Are Ryzens really still susceptible? I thought they were already immune?
Thanks for the tip. Actually MSI was going to be my first choice any way.
It's not really much cheaper than an E3 is it? The 1700 cost about US$300, and an E3-1230v6 is about $275.
The board may be a little cheaper, depending on model, but if using ECC, the RAM is the same too. So Ryzen doesn't look to be cheaper. Though the specs look good, so maybe you get more bang for your buck?
Yeah but 'Xeon' makes it look and sound professional
. Who cares that an i7 is just as good and cheaper? It's marketed for consumers, and that doesn't look good for Businessess. It's kind of like VWs, Audi, Seat and Skoda. All the same stuff, just different prices.
@randvegeta
Again, I'm not opposed to your approach. A ryzen based system isn't really "unprofessional"; it's just not targeted at servers and lacks some of the EPYC features. I'm also not that concerned about the lack of ipmi mainboards because looking at the price differences and price/features ratio it's not that bad a way to simply use an external (or pcie) kvm solution.
Plus, and most importantly, we're talking at LET and not at "high end super professional hosting".
An X-370 based board gives you a very decent processor option, 64 GB (maybe more, dunno, didn't check) memory and plenty pcie. And 8 or so sata ports, too, btw. So, if you're thinking about budget VMs you are certainly not misguided imo when you look at Ryzen.
And while you don't get mainboards w/ipmi having plenty of pcie lanes offers some attractive options which might serve well as market differentiators in your segment.
As for S/M: There are 2 sides to that. a) the end user/customer side. There, very many believe that AMD is safe which obviously is an advantage for you. b) the technical reality. There the truth is that zen isn't concerned about 1 out of 3 vulnerabilities/attack paths. So you're better than with intel anyway. Plus, and more importantly, I expect to see a real and full solution from amd quite a bit sooner than from intel.
On a sidenote, I'd suggest to go with the 1700 (low tdp, low price) rather than with the bigger ones. For a start 8 3GHz cores aren't lousy at all; moreover lower tdp ~ higher reliability and life times and less power costs.
Yes that was my thinking also. 65TDP is even better than most of the Intel Xeon E3 line up. The lower base clock speed is more than made up for by having double the coures, and 2.5x more cache. The bench score is also around 20% better than a similarly priced E3 (although that doesn't mean much in the server world).
Because these things have 8 cores, you can offer a very decent dedicated core with 2 - 4GB RAM depending on if you have 32 or 64GB RAM installed in the node. It actually sounds pretty decent. With the E3 v5/v6 lineup, dedicated cores would be considerably more expensive, and have seemingly unreasonably large amount of RAM.
The Ryzen seems more in line with an E5, but with higher clock speeds and lower price. On paper, it looks like lower power consumption too, but I won't know for sure till I put them into production.
Seems no one has anything bad to say about Ryzen, and although they are not really my first choice, it will be interesting to see how they compare to our traditional E3/E5 lineup.
If you're not willing to wait for EPYC then have a look at Threadripper. Since they don't have dual socket AM4 motherboards yet, Ryzen is basically limited to 8 cores with 4 sticks of RAM while a Threadripper motherboard will get you up to 16 cores and 8 sticks of RAM. One other thing to note is that no AM4/sTR4 motherboards have onboard video and the CPUs don't have integrated graphics either so expect to pick up a videocard of some sort unless you buy a remote management card.
Can you recommend a management card?
I don't think the Threadripper is a good option. They cost 3x more than an 1700, and use more than twice as much power . The 1700 seems to be the best value.
I don't know of any management cards off hand. As for the value, that's subjective of course. Either way I would wait for the EPYC CPUs to be available before deciding.
I wouldn't expect the Ryzen refresh (Zen+) to contain any Spectre mitigations. It's too soon. Zen 2 at the earliest is when I would expect for hardware mitigations, so maybe next year.
As I said, the need for new machines deployed is fairly urgent. We could order some EPYC servers, but the delivery time will be weeks, and we need machines before the end of the week.
As for the value, these boxes are intended to be used for virtualisation. No single VM would be allocated so much of the CPU resources that a single 1700 cannot handle it.
The way I look at it is pretty simple.
A Ryzen 7 1700 CPU is about US$300, and these desktop boards are apparently pretty cheap (under $100).
A Threadripper cost $900 - $1,000 (about 3-3.5x more) but only has 12 or 16 threads. So more than 3 x the price for around 1.8 - 2.2 x the performance. The motherboards are a good 4 times the price too.
Since RAM and Disk would be the same (per VM), there are no savings here. so on a cost per VM basis, the 1700 looks better. The benefit of the Threadripper over the 1700 is greater density. But I'm in no need for any great density, and 64GB on a single socket CPU is more than reasonable. The power consumption on the Threadripper also looks to be around double too, so the performance per watt is probably around the same. Another benefit of greater density I guess would be the fact you only need 1 chassis and 1 PSU instead of 2 (assuming you consider a single Threadhripper equivalent to 2 R7s). But that is not an issue for us either.
Any way, long story short, the point is that for the purpose of running VMs, the 1700s look better value unless you are really short on rack space and need the higher density.
PC Weasel 2000
Probably right about VMs. There are still times it's hard to beat the attraction of a monster dedi like the Epyc lets you build (dunno about Threadripper). Maybe someday.
Nope. Don't! Those sub 100$ mobos are B-250 based. I strongly suggest to go with a X-370 based one which are about +50$ (but be careful to avoid the purely gamer boards).
Another hint: Don't fall for the wild "expert" stories about memory + high clocking. What you'll want for hosting is 3 or 3.2 GHz rated 15,35 memory and a mem. setup of 2.8 GHz in the bios.
Reason: memory speed really dictates some proc. internal bus speeds, hence you don't want the standard 2133 but higher. At the same time (at least the current gen. of) mobos tend to get a little unreliable if you use 100% of the memories theoretical speed. 2.8 GHz seems to be the sweet spot for 24/ running boxen; fast but solid and reliable. (and again: be sure to get 15,35 cycles memory, most are 15,36 or 16,36)
Right! Thanks for the reminder.
If you put up some competitively priced Ryzen dedis with ECC, I can't promise anything but I'll be paying close attention.
struggling to find details on boards which support ecc or not.
Mine does. Asus X-370 pro. But I bet msi has such boards, too. From what I see the problem is that ecc is not an important feature in that market segment, so the manufacturers don't advertise it and the "reviewers" usually don't mention it.
But the X-370 chipset supports it and I see no reason why a manufacturer would go an extra step to remove it.
I guess you'll have to look at the specs of any mobo you're interested in. That's where it's usually mentioned.
I have some DDR4 ECC UDIMMs off hand. My Supermicro boards for my E3 v5/v5 servers oddly support non-ECC, even though the specs say they don't.
I'll use my existing ECC sticks and test if they work on the X-370 chip boards. Would be interesting.
But just because it can take ECC sticks, does it actually use the ECC functionality?
Didn't test but wouldn't know a fucking reason why they wouldn't. Besides, keep in mind that that functionality is often provided in the dma controller which simply signals an error to the cpu.