All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
VyOS Software Router - What kind of performance can be expected?
I'm looking at using a number of DIY routers, not so much to save money, but for the convenience and flexibility.
I am familiar with Vyatta and Vyatta based routers and am happy with the features, functionality and stability it is generally associated with. What I'm not sure about is realistic performance that can be expected for given hardware.
I am using Intel Pro Gbit NICs.
1 router will be built using C2D E8400 (3.0GHz Dual Core).
Another 2 will be built using Core i3-3220 (3.3Ghz Dual Core).
I also have a Dell C1100 (Dual Xeon L5520 - onboard NIC) that I will use for spinning up virtual routers on demand.
I am wondering what kind of routing performance I may get from these machines. As I understand, clock speed is more important than cores, as well as decent NICs. So in terms of routing capacity, what is realistic? Would the E8400 and i3-3220 be able to handle in excess of 2Mpps? For the dedicated machines, I expect transfer rates under 1G (typically in the region of 50 - 100Mbit, and burst to 1G). And for the virtual routers, I expect rates of under 100M, typically in the 20 - 50Mbit range.
I plan to do BGP on most of the routers, so all routers will have 4GB + RAM to handle multiple full tables.
Anyone with similar experience to this kind of setup? How would these compare to say a Juniper J2320 router, UBNT Edgerouter lite, or UBNT Edgerouter Pro.
Is it worth doing or are the 'hardware' (I know the above examples are still SW) routers still a better choice?
Thanks in advance.
Comments
VyOS is a solid choice for routing software. 2Mpps will probably not be attainable on the i3. Its a possibility on the Dual 5520 if you're not applying any routing policies, but would vary depending on the type of traffic being thrown at it (Random source syn generates more load than a few single source streams, for example). If you're looking to do 2MPPS, try an E3 with intel based network adapters. That should give you sufficient power to route at that level.
If you're looking at budget routing hardware, try the Ubnt ER Pro or Mikrotik 1036-8G-2s+/1072-1g-8s+. I can't speak to the ER Pro, although I've heard good things. The Mikrotik CCRs will do fine if they are configured correctly (very important).
Ubiquiti ER8 is same as Ubiquiti EdgeRouter Pro just without the SFP ports, which is great if you want to save some money and you don't need SFP (like the OP doesn't)... just a thought.
EdgeRouter Lite has too few RAM for a full BGP table, you'd need to go with an EdgeRouter Pro or ER-8. They are nice hardware but if you want to use certain features (like complicated policy based routing, bridging between interfaces, ...) the hardware acceleration shuts itself off.
Doesn't the pro also have a slightly faster CPU for the non offloaded stuff, or am I just imagining that?
Correct, the data sheet does mention that.
Edgerouter: Dual-Core 800 MHz, MIPS64
Edgerouter Pro: Dual-Core 1 GHz, MIPS64
NIC is most important part, but go with Ubnt to save electric, or VyOS, Vyatta is a dead. I don't know about others, talking 2-3Mbps limit, I had a VyOS E3 with a HP(Mellanox) dual port 10G that I was able to achieve full wire speed. I also have an old Atom still in use after many years, full BGP with multiple views
I would update the Atom to VyOS, but the autonull script breaks badly on VyOS
EDIT: I was wrong, others talking out 2Mpps, yeah, without a quality NIC it is doubtful to attain. Just avoid anything with Realtec, 700Mbps limit from the 1Gbit ports.
Thanks for the input everyone.
I'm familiar with the UBNT products and have been pretty happy with them, but they arent as power frugal as I once thought they were. They use about the same amount of power as an efficient entry level server (like an i3).
I'm not dead set on achieving 2Mpps, but I am hoping to be competitive in with the ER8 Pro. I probably will be using these as very simple routers with minimal policies. But I am surprised that you think the L5520 would be any good. I know they have 8 cores, but can VyOS make use of multiple cores for routing? Isn't the low clock speed a hindrance?
What about using the L5520s for virtual routers? As I have been thinking that VyOS can only really use 1 core for routing, I figure the Dual L5520 would be good for at least 8 virtual routers. Bad idea? Virtual routers perform poorly?
Sorry, I totally forgot about that. But you can upgrade RAM on ER8/ERPro by opening it up and putting in a 4GB DDR3 SODIMM instead of the 2GB it comes with.
I've once run VyOS as an internal router on an Opteron 1354 (Dacentec) (KVM). During testing, it ran at full speed (~600mbps down, ~100mbps up). One CPU was assigned to it, and it did not even use all of the CPU. The main issue was with the RAM, despite previously using pfSense on 512mb, VyOS required 1G. However, it may have mainly been due to the fact that I had lots of IP lists running around, though pfSense did to, and it never used anything near 512MB.
Decent NICs are critical, you don't want an interrupt storm as that will kill the performance.
Intel X520's are quite good for PC routers
Look at the NIC chipset to find out how many PPS it will support and critically at what size. Some benchmarks show undersized packets to boost the PPS.
PCIv3 helps too
But any DDOS is going to take down the soft router hard.
The routers will be behind other edge routers, and in HK, where these will be used, probably the DDOS will flood our network before crashing the routers! We have subcribed to DDoS-Guard's network protection service in such instances to ensure we remain online.
Any way, as these x86 routers will be used behind edge routers, have plenty of ram, and use nice Intel Pro NICs, I am hopeful they will be able to handle what I have planned for them. Will just have to wait and see!