Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


VyOS Software Router - What kind of performance can be expected?
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

VyOS Software Router - What kind of performance can be expected?

randvegetarandvegeta Member, Host Rep

I'm looking at using a number of DIY routers, not so much to save money, but for the convenience and flexibility.

I am familiar with Vyatta and Vyatta based routers and am happy with the features, functionality and stability it is generally associated with. What I'm not sure about is realistic performance that can be expected for given hardware.

I am using Intel Pro Gbit NICs.

1 router will be built using C2D E8400 (3.0GHz Dual Core).
Another 2 will be built using Core i3-3220 (3.3Ghz Dual Core).

I also have a Dell C1100 (Dual Xeon L5520 - onboard NIC) that I will use for spinning up virtual routers on demand.

I am wondering what kind of routing performance I may get from these machines. As I understand, clock speed is more important than cores, as well as decent NICs. So in terms of routing capacity, what is realistic? Would the E8400 and i3-3220 be able to handle in excess of 2Mpps? For the dedicated machines, I expect transfer rates under 1G (typically in the region of 50 - 100Mbit, and burst to 1G). And for the virtual routers, I expect rates of under 100M, typically in the 20 - 50Mbit range.

I plan to do BGP on most of the routers, so all routers will have 4GB + RAM to handle multiple full tables.

Anyone with similar experience to this kind of setup? How would these compare to say a Juniper J2320 router, UBNT Edgerouter lite, or UBNT Edgerouter Pro.

Is it worth doing or are the 'hardware' (I know the above examples are still SW) routers still a better choice?

Thanks in advance.

Comments

  • VyOS is a solid choice for routing software. 2Mpps will probably not be attainable on the i3. Its a possibility on the Dual 5520 if you're not applying any routing policies, but would vary depending on the type of traffic being thrown at it (Random source syn generates more load than a few single source streams, for example). If you're looking to do 2MPPS, try an E3 with intel based network adapters. That should give you sufficient power to route at that level.

    If you're looking at budget routing hardware, try the Ubnt ER Pro or Mikrotik 1036-8G-2s+/1072-1g-8s+. I can't speak to the ER Pro, although I've heard good things. The Mikrotik CCRs will do fine if they are configured correctly (very important).

  • shovenoseshovenose Member, Host Rep

    @dot_txt said:
    VyOS is a solid choice for routing software. 2Mpps will probably not be attainable on the i3. Its a possibility on the Dual 5520 if you're not applying any routing policies, but would vary depending on the type of traffic being thrown at it (Random source syn generates more load than a few single source streams, for example). If you're looking to do 2MPPS, try an E3 with intel based network adapters. That should give you sufficient power to route at that level.

    If you're looking at budget routing hardware, try the Ubnt ER Pro or Mikrotik 1036-8G-2s+/1072-1g-8s+. I can't speak to the ER Pro, although I've heard good things. The Mikrotik CCRs will do fine if they are configured correctly (very important).

    Ubiquiti ER8 is same as Ubiquiti EdgeRouter Pro just without the SFP ports, which is great if you want to save some money and you don't need SFP (like the OP doesn't)... just a thought.

  • randvegeta said: UBNT Edgerouter lite, or UBNT Edgerouter Pro.

    EdgeRouter Lite has too few RAM for a full BGP table, you'd need to go with an EdgeRouter Pro or ER-8. They are nice hardware but if you want to use certain features (like complicated policy based routing, bridging between interfaces, ...) the hardware acceleration shuts itself off.

  • @shovenose said:
    Ubiquiti ER8 is same as Ubiquiti EdgeRouter Pro just without the SFP ports, which is great if you want to save some money and you don't need SFP (like the OP doesn't)... just a thought.

    Doesn't the pro also have a slightly faster CPU for the non offloaded stuff, or am I just imagining that?

  • BoltersdriveerBoltersdriveer Member, LIR
    edited February 2016

    dragon2611 said: Doesn't the pro also have a slightly faster CPU for the non offloaded stuff, or am I just imagining that?

    Correct, the data sheet does mention that.

    Edgerouter: Dual-Core 800 MHz, MIPS64
    Edgerouter Pro: Dual-Core 1 GHz, MIPS64

  • miTgiBmiTgiB Member
    edited February 2016

    randvegeta said: As I understand, clock speed is more important than cores

    NIC is most important part, but go with Ubnt to save electric, or VyOS, Vyatta is a dead. I don't know about others, talking 2-3Mbps limit, I had a VyOS E3 with a HP(Mellanox) dual port 10G that I was able to achieve full wire speed. I also have an old Atom still in use after many years, full BGP with multiple views

    $ free -m
                 total       used       free     shared    buffers     cached
    Mem:          3041       2111        930          0         31         80
    -/+ buffers/cache:       2000       1041
    Swap:            0          0          0
    
    model name      : Intel(R) Atom(TM) CPU D525   @ 1.80GHz
    
    $ uptime
     07:19:28 up 298 days,  7:04,  2 users,  load average: 0.00, 0.03, 0.05
    $ show version
    Version:      VC6.4-2012.05.31
    

    I would update the Atom to VyOS, but the autonull script breaks badly on VyOS

    EDIT: I was wrong, others talking out 2Mpps, yeah, without a quality NIC it is doubtful to attain. Just avoid anything with Realtec, 700Mbps limit from the 1Gbit ports.

    Thanked by 1Kris
  • randvegetarandvegeta Member, Host Rep

    Thanks for the input everyone.

    I'm familiar with the UBNT products and have been pretty happy with them, but they arent as power frugal as I once thought they were. They use about the same amount of power as an efficient entry level server (like an i3).

    dot_txt said: Its a possibility on the Dual 5520 if you're not applying any routing policies

    I'm not dead set on achieving 2Mpps, but I am hoping to be competitive in with the ER8 Pro. I probably will be using these as very simple routers with minimal policies. But I am surprised that you think the L5520 would be any good. I know they have 8 cores, but can VyOS make use of multiple cores for routing? Isn't the low clock speed a hindrance?

    What about using the L5520s for virtual routers? As I have been thinking that VyOS can only really use 1 core for routing, I figure the Dual L5520 would be good for at least 8 virtual routers. Bad idea? Virtual routers perform poorly?

  • shovenoseshovenose Member, Host Rep

    @dragon2611 said:
    Doesn't the pro also have a slightly faster CPU for the non offloaded stuff, or am I just imagining that?

    Sorry, I totally forgot about that. But you can upgrade RAM on ER8/ERPro by opening it up and putting in a 4GB DDR3 SODIMM instead of the 2GB it comes with.

  • @randvegeta said:
    I'm looking at using a number of DIY routers, not so much to save money, but for the convenience and flexibility.

    I am familiar with Vyatta and Vyatta based routers and am happy with the features, functionality and stability it is generally associated with. What I'm not sure about is realistic performance that can be expected for given hardware.

    I am using Intel Pro Gbit NICs.

    1 router will be built using C2D E8400 (3.0GHz Dual Core).
    Another 2 will be built using Core i3-3220 (3.3Ghz Dual Core).

    I also have a Dell C1100 (Dual Xeon L5520 - onboard NIC) that I will use for spinning up virtual routers on demand.

    I am wondering what kind of routing performance I may get from these machines. As I understand, clock speed is more important than cores, as well as decent NICs. So in terms of routing capacity, what is realistic? Would the E8400 and i3-3220 be able to handle in excess of 2Mpps? For the dedicated machines, I expect transfer rates under 1G (typically in the region of 50 - 100Mbit, and burst to 1G). And for the virtual routers, I expect rates of under 100M, typically in the 20 - 50Mbit range.

    I plan to do BGP on most of the routers, so all routers will have 4GB + RAM to handle multiple full tables.

    Anyone with similar experience to this kind of setup? How would these compare to say a Juniper J2320 router, UBNT Edgerouter lite, or UBNT Edgerouter Pro.

    Is it worth doing or are the 'hardware' (I know the above examples are still SW) routers still a better choice?

    Thanks in advance.

    I've once run VyOS as an internal router on an Opteron 1354 (Dacentec) (KVM). During testing, it ran at full speed (~600mbps down, ~100mbps up). One CPU was assigned to it, and it did not even use all of the CPU. The main issue was with the RAM, despite previously using pfSense on 512mb, VyOS required 1G. However, it may have mainly been due to the fact that I had lots of IP lists running around, though pfSense did to, and it never used anything near 512MB.

  • Decent NICs are critical, you don't want an interrupt storm as that will kill the performance.

    Intel X520's are quite good for PC routers

    Look at the NIC chipset to find out how many PPS it will support and critically at what size. Some benchmarks show undersized packets to boost the PPS.

    PCIv3 helps too

    But any DDOS is going to take down the soft router hard.

  • randvegetarandvegeta Member, Host Rep

    MarkTurner said: But any DDOS is going to take down the soft router hard.

    The routers will be behind other edge routers, and in HK, where these will be used, probably the DDOS will flood our network before crashing the routers! We have subcribed to DDoS-Guard's network protection service in such instances to ensure we remain online.

    Any way, as these x86 routers will be used behind edge routers, have plenty of ram, and use nice Intel Pro NICs, I am hopeful they will be able to handle what I have planned for them. Will just have to wait and see!

Sign In or Register to comment.