Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Best Core/Edge Router
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Best Core/Edge Router

I'm looking for the best router for 3x1G Fiber uplinks and 1x10G fiber uplink basically a good core router. I don't care about the brand just needs to be durable + full BGP table.

Thanked by 1ValiSXP
«1

Comments

  • ClouviderClouvider Member, Patron Provider

    MX240, failing that MX204/MX104 if small number of feeds. Nothing below that.

    Thanked by 1techhelper1
  • First-RootFirst-Root Member, Host Rep

    Agree to @Clouvider . @jack mx204 is a fixed Router with a single routing engine, mx104 is modular and can have redundant routing engines

  • ClouviderClouvider Member, Patron Provider
    edited October 2018

    Each use case is different. OP asked for good routers not the cheapest routers, we don’t know OPs preference re: multiple REs or multiple single RE routers, or whether the OP needs replaceable linecards/PFEs or room for further expansion, so I put in 3 options satisfying the requirement ‘good’, support for full table and 10G so the OP has something to choose from. :-)

    Thanked by 1First-Root
  • First-RootFirst-Root Member, Host Rep
    edited October 2018

    @Jack said:

    @FR_Michael said:
    Agree to @Clouvider . @jack mx204 is a fixed Router with a single routing engine, mx104 is modular and can have redundant routing engines

    You could probably buy 5 204s for a fully spec’d 104.

    Remember that you need to buy a JunOS license for the mx204.

  • FHRFHR Member, Host Rep

    Depending on the exact requirements (and price), an 1U machine with a 10 gig card might be perfectly good.

  • Hi

    You can try a Mikrotik router. CCR1072-1G-8S+ has 8xSFP+ cages plus Gigabit Ethernet for management. It has 72 core at 1 GHz, 16 GB RAM. I use those routers for more than 7 years. No failure.

    Thanked by 1HashTag
  • @FHR said:
    Depending on the exact requirements (and price), an 1U machine with a 10 gig card might be perfectly good.

    How so?

  • ZareZare Member, Host Rep

    @HashTag said:

    @FHR said:
    Depending on the exact requirements (and price), an 1U machine with a 10 gig card might be perfectly good.

    How so?

    You can use a 1U SuperMicro server with a decent CPU/RAM, SSD's in RAID, dual PSU and dual 10G network card. Then just run a router software on it and your done. This is the cheapest option.

    For enterprise i'd agree with Dom and suggest the MX range of routers. But they aren't cheap.

    Thanked by 3Clouvider HashTag FHR
  • ClouviderClouvider Member, Patron Provider

    If you go the software route just make sure you use a network card generating low number of interrupts. Intel is quite recommended here.

  • FHRFHR Member, Host Rep
    edited October 2018

    Zare said: You can use a 1U SuperMicro server with a decent CPU/RAM, SSD's in RAID, dual PSU and dual 10G network card. Then just run a router software on it and your done. This is the cheapest option.

    You don't need much RAM and even slow HDDs will suffice. It's all about quality of NICs - don't even attempt to route on cheapo cards. BCM5719 based stuff (found in HP DL360 Gen8 - HP331FLR) will, from my testing, crash at ~300k packets per second. And by crash I mean really crash - system will stop seeing it for a while.

  • AlexBarakovAlexBarakov Patron Provider, Veteran

    @Clouvider said:
    If you go the software route just make sure you use a network card generating low number of interrupts. Intel is quite recommended here.

    And CPUs with as high single thread performance as possible. And still will most likely never gonna be able to sustain over 1.5M pps.

    Thanked by 2HashTag Clouvider
  • Actually, it is quite possible to get over 14Mpps when using networking frameworks that bypass the kernel, like netmap, DPDK, etc...

    https://github.com/Gandi/packet-journey

    https://github.com/luigirizzo/netmap

    The best part is when they integrate netlink support for the Linux kernel, then any standard routing software works. In fact, that's how whitebox switches like Cumulus Networks and others work. They let the modules burn the info into the switch ASIC.

    AlexBarakov said: And CPUs with as high single thread performance as possible. And still will most likely never gonna be able to sustain over 1.5M pps.

    This is why you spread the RX queues over multiple cores. It also helps if you have multiple CPUs so that each card can still have full performance.

    @ValiSXP said:
    Hi

    You can try a Mikrotik router. CCR1072-1G-8S+ has 8xSFP+ cages plus Gigabit Ethernet for management. It has 72 core at 1 GHz, 16 GB RAM. I use those routers for more than 7 years. No failure.

    Just no. I'm pretty sure they're still crap when it comes to calculating the routes on initial startup or if a link flaps.

    Clouvider said: MX240, failing that MX204/MX104 if small number of feeds. Nothing below that.

    This is the absolute truth.

    Thanked by 1jh_aurologic
  • HashTagHashTag Member
    edited October 2018

    @Zare said:

    @HashTag said:

    @FHR said:
    Depending on the exact requirements (and price), an 1U machine with a 10 gig card might be perfectly good.

    How so?

    You can use a 1U SuperMicro server with a decent CPU/RAM, SSD's in RAID, dual PSU and dual 10G network card. Then just run a router software on it and your done. This is the cheapest option.

    For enterprise i'd agree with Dom and suggest the MX range of routers. But they aren't cheap.

    @techhelper1 said:
    Actually, it is quite possible to get over 14Mpps when using networking frameworks that bypass the kernel, like netmap, DPDK, etc...

    https://github.com/Gandi/packet-journey

    https://github.com/luigirizzo/netmap

    The best part is when they integrate netlink support for the Linux kernel, then any standard routing software works. In fact, that's how whitebox switches like Cumulus Networks and others work. They let the modules burn the info into the switch ASIC.

    AlexBarakov said: And CPUs with as high single thread performance as possible. And still will most likely never gonna be able to sustain over 1.5M pps.

    This is why you spread the RX queues over multiple cores. It also helps if you have multiple CPUs so that each card can still have full performance.

    @ValiSXP said:
    Hi

    You can try a Mikrotik router. CCR1072-1G-8S+ has 8xSFP+ cages plus Gigabit Ethernet for management. It has 72 core at 1 GHz, 16 GB RAM. I use those routers for more than 7 years. No failure.

    Just no. I'm pretty sure they're still crap when it comes to calculating the routes on initial startup or if a link flaps.

    Clouvider said: MX240, failing that MX204/MX104 if small number of feeds. Nothing below that.

    This is the absolute truth.

    What do you guys recommend in terms of a server? Ill go ahead and build one for this purpose.

  • AnthonySmithAnthonySmith Member, Patron Provider

    Netgear DG834

    :)

  • @AnthonySmith said:
    Netgear DG834

    :)

    O You funny guy.

    I was actually thinking of doing an HP server because the ILO and using

    HP 560SFP+ Dual Port DA/SFP

  • randvegetarandvegeta Member, Host Rep

    HashTag said: What do you guys recommend in terms of a server? Ill go ahead and build one for this purpose.

    Something like a Xeon E3, 8GB RAM, and SSD or high quality USB stick to load the OS onto, and Intel X540 T2 NICs.

    Get the dual port NICs, not the single ones. Get a few NICs, and that should be good enough for a few gig and a few MPPS.

    I've got a couple of these in live deployment. Easily handling 5G+ of sustained traffic.

    But you probably want at least 2 of these configured with VRRP for redundancy.

    It's a pretty effective, low cost and very flexible router solution. You don't need particulalrly new CPUS either. An E3v2 or v3 is plenty powerful enough. The v6 models are only marginally more powerful. E5s tend to cost A LOT more and have much lower clock speed. And actually high clock speeds are important.

    These DIY options will run circles around the Juniper (or any big brand) CPU based routers, and cost you a fraction as much. So with the savings, you may as well setup extra routers for redundancy.

    Thanked by 1HashTag
  • ClouviderClouvider Member, Patron Provider
    edited October 2018

    What ? Are you implying that Linux based or MikroTik based router running in an E3 will outperform several times (run circles around) say a Juniper vMX running in comparable hardware? Did you ever had a chance to test them? There is absolutely no chance you can get anywhere close using a Linux kernel, just forget about it.

    OP if you have a budget or more than 1G sustained traffic and you don’t want to be taken down with a smallish DDoS with a lot of packets I highly recommend that you go the hardware route.

    Thanked by 2HashTag jh_aurologic
  • randvegetarandvegeta Member, Host Rep

    Clouvider said: or MikroTik based router

    Not Mikrotik,

    say a Juniper vMX running in comparable hardware?

    Comparable hardware? The point is that you pay much much more for the same performance if you're going for a so called 'professional' router. To be fair, it has been some years since I touched a Juniper CPU based router, but it fell over with very modest traffic (not even DDoS).

    I have deployed several E3 based routers, and CPUs can handle several G of attack traffic just fine, and that's with Netflow running with high sampling rate. Turn that off, and it basically idles when multiple 1G links are maxed out.

    I'm not sure how well they work with multiple 10G uplinks, but these DIY routers can more than handle a DDoS if the links are mostly 1G. You fill up the pipe long before the CPU starts to complain. And since you can build these on a budget, I don't see the problem.

    The idea that a 'smallish DDoS attack with a lot of packets' will cause performance problems is really just not true. A decent setup can handle several MPPS, and that's enough for a few G. Again.. I haven't tested on multiple 10G, so maybe sub 10G is the limit. But for the price, they cant be beaten.

    For the same amount of money, you get far worse performance with any branded CPU based router. And for the same performance, any branded CPU router cost orders of magnitude more.

    Thanked by 1HashTag
  • Decided to go with a route that I know and has a GUI the Ubiquiti EdgeRouter Infinity 10-Gigabit SFP+ EdgeRouter (ER-8-XG). Going to see how this works out for me going into networking for the first time. I have a 30 days warranty/return so if something comes up ill just return it. Hopefully it will be $1,450 well spent.

  • randvegetarandvegeta Member, Host Rep

    HashTag said: Decided to go with a route that I know and has a GUI the Ubiquiti EdgeRouter Infinity 10-Gigabit SFP+ EdgeRouter (ER-8-XG).

    I don't have experience with the new 10G routers they sell, but the ER8Pro performs no where near as well as advertised. It CANNOT handle 2MPPS, despite it's claims.

    I imagine that the 10G model is probably better, but I suspect probably also suffers from the same kind of limitations.

    I mean, you may as well just go VyOS. With that in mind, if you get hit with a DOS and it holds up, I'd be interested in knowing.

    Thanked by 1HashTag
  • @randvegeta said:

    HashTag said: Decided to go with a route that I know and has a GUI the Ubiquiti EdgeRouter Infinity 10-Gigabit SFP+ EdgeRouter (ER-8-XG).

    I don't have experience with the new 10G routers they sell, but the ER8Pro performs no where near as well as advertised. It CANNOT handle 2MPPS, despite it's claims.

    I imagine that the 10G model is probably better, but I suspect probably also suffers from the same kind of limitations.

    I mean, you may as well just go VyOS. With that in mind, if you get hit with a DOS and it holds up, I'd be interested in knowing.

    I'm interested to see as well how it holds up.

  • Clouvider said: What ? Are you implying that Linux based or MikroTik based router running in an E3 will outperform several times (run circles around) say a Juniper vMX running in comparable hardware? Did you ever had a chance to test them? There is absolutely no chance you can get anywhere close using a Linux kernel, just forget about it.

    VyOS is regular Linux forwarding.

    The packet journey project that I linked in my last post can handle 20G's of 64-byte packets since bypasses the kernel with DPDK. Which would be more than enough to handle most DDoS attacks and trigger a null route if needed.

    Chelsio in fact has NICs and drivers that can handle 75Mpps on a single socket E5 down to 64 byte packets, which equates to about 52Gbits line speed at the very worse case scenario. (https://www.chelsio.com/wp-content/uploads/resources/t6-100g-dpdk-linux.pdf) I'm sure no one here would ever come close to hitting that but hang a Juniper QFX5200-48S or equivalent off of it to get a bunch of 10G ports, and its a really solid contender. 40G would be too easy to implement.

    In the end it's really what you know, and how you choose to implement it.

  • jsgjsg Member, Resident Benchmarker

    @Clouvider said:
    What ? Are you implying that Linux based or MikroTik based router running in an E3 will outperform several times (run circles around) say a Juniper vMX running in comparable hardware? Did you ever had a chance to test them? There is absolutely no chance you can get anywhere close using a Linux kernel, just forget about it.

    Juniper runs/JunOs is basically BSD based, which in some points is a bit better than linux and in others is a bit worse.

    I also advise to be careful there because of the processors. Most - even large - routers don't have powerfull processors (say Xeon) and those that do have it for their own reasons the usual one being a form of idiocy ("Just use a Xeon and linux to keep things simple"). In a real router (TM) the control plane doesn't need to be very powerful (which is why Arms are found even in higher end equipment nowadays). The data plane is where power is needed.

    Looking at the concrete case here we're not even talking about something that would in any way be big. A smartphone processor plus decent network cards would already be overkill.
    One important point is see is the question of reliability/availability. Any PC based routers can or should only be employed where "oh well, the gateway is down till tomorrow" wouldn't be a problem.
    That's where real routers enter the game. Redundancy is the well established way to go but that's something only professional routers can offer (And no, having two cheap PC-based routers in some kind of fall-back config often is not comparable or viable).

    In summary I'd go along with @Clouvider's advice but there's a but: Juniper (just like Cisco) have been quite sloppy wrt. security but unpleasant as that maybe you won't get much better (more secure) equipment anywhere because almost everyone incl. linux and the BSDs have grave problems lurking and waiting to blow up. But the (except entry toys) Juniper seem to be among the best choices.

    Oh and: it would certainly not be a disadvantage to have an OpenFlow enabled box.

  • jsg said: Juniper runs/JunOs is basically BSD based, which in some points is a bit better than linux and in others is a bit worse.

    Anything that has to pass through a kernel is going to be slow, it's simple as that.

    jsg said: Any PC based routers can or should only be employed where "oh well, the gateway is down till tomorrow" wouldn't be a problem.

    If you're talking about big iron replacements, then sure, but for home use, pfSense, OPNsense, any Linux distro, or even Windows can be a simple NAT firewall router. I've done it all.

    I will admit that there are providers here that use Mikrotik or regular servers for routing. It's to each their own preference.

    Thanked by 1jh_aurologic
  • jh_aurologicjh_aurologic Member, Patron Provider

    Kernel Bypass techniques as described by @techhelper1 are the framework of most modern packet analyzers, ddos filtrations and correctly implemented software routers.

    Back in 2013, we "played" with iptables and self written kernel modules to filter out ddos attacks. At some point, it was just to slow and unstable to keep up with large attacks and we would have needed a rack full of servers, where one box was only able to deal with 4,5mpps, to filter out the attacks we deal today.

    That was the reason for us, to implement everything into a multithreaded userspace application. All the traffic (rx+tx) bypasses the kernel completely with the help of netmap - see http://info.iet.unipi.it/~luigi/netmap/ at a very decent speed / load proportion.

    I wouldnt recommend to process traffic within the kernel, as soon as you serve customers or have to deal with either in- or outbound ddos attacks.

  • ClouviderClouvider Member, Patron Provider

    Netmap can’t do routing in the traditional sense, especially when BGP comes to play.

  • randvegetarandvegeta Member, Host Rep

    jsg said: One important point is see is the question of reliability/availability. Any PC based routers can or should only be employed where "oh well, the gateway is down till tomorrow" wouldn't be a problem.

    Can I ask why exactly you would have this view? Perhaps we are thinking of different things... different scenarios.

    But why exactly can you not get sufficient redundancy using cheap commodity gear?

    It seems software based redundancy solutions is the way forward, and the direction we seem to be going in to achieve high levels of reliability and scalability at cost effective rates.

    Using 2 or more commodity server based routers should be able to provide pretty high level of reliability between them if configured correctly.

  • jmginerjmginer Member, Patron Provider
    edited October 2018

    With a dual E5-2667 (+3.00 Ghz per core) you can do BGP with 3-4 upstreams and surpass 5-6 Gbps without problems.

    https://mum.mikrotik.com/presentations/EU18/presentation_5188_1524562405.pdf

    Thanked by 1ValiSXP
  • jsgjsg Member, Resident Benchmarker
    edited October 2018

    @techhelper1 said:

    Anything that has to pass through a kernel is going to be slow, it's simple as that.

    Wrong department; same for linux' kernel bypass model.

    In a real router(TM) - which today means n x 10Gb backend or higher -and- use typically in a DC - (whatever) kernel is little to do with the packet shifting. In a real router the kernel is in the control plane and the packets are dealt with in the data plane which typically is an FPGA or ASIC.

    Imagine it roughly like this: packet enters, data plane does all grunt work; control plane then only gets asked for the out port ("routing"). In higher end systems even that isn't done by asking the control plane but the (at least the most used) routes are pushed into the data plane.

    With (let's call them) software routers like pfsense that DP/CP differentiation is (almost) non existant. Everything is done by one system and in software.
    Trying to squeeze out more performance lead to two major approaches, (a) put at least some DP capabilities into smart network adapters, and (b) kernel bypassing.

    @randvegeta said:

    jsg said: One important point is see is the question of reliability/availability. Any PC based routers can or should only be employed where "oh well, the gateway is down till tomorrow" wouldn't be a problem.

    Can I ask why exactly you would have this view? Perhaps we are thinking of different things... different scenarios.

    But why exactly can you not get sufficient redundancy using cheap commodity gear?

    It's in the details.

    Simple failover solutions (linux, BSD based PCs) (a) have too long cycles and (b) basically are just "let the other box take over" solutions. That's OK for private and some (many?) businesses but not in the network core (e.g. data centers).

    To improve (a) one needed to massively increase house keeping traffic and to improve (b) one needed to mirror much more state - and keep in mind that all that is software based.

    The details I spoke about are mainly (a) the connection (between the 2 or more systems) and (b) the state mirroring depth, both of which are related.
    In a real router the connection is one or more Serdes lines (no kernel, no linux, etc) and the state mirroring is (a) deep and (b) mostly between the 2 data planes. The fail over is merely a "which one is active" message.

    Don't underestimate the details. To transfer a say 1 byte control message via linux/ethernet is in the hundreds of microseconds, while doing the same on the hardware/data plane level is in the tens of nanoseconds.

    For home or office use being basically offline for a milliseconds and potentially breaking existing connections/sessions is acceptable and usually being taken care of by the TCP stack. In a data center though that's not acceptable.

    Maybe I was mistaken but I took OP to ask about a (possibly small) ISP or DC situation, hence my response.

Sign In or Register to comment.