News on 40 Gbps software router:
http://www.technologyreview.in/communications/26096/

100 Gbps software router explained:
http://www.google.com/url?sa=t&source=w ... bw&cad=rja

40 Gbps Packetshader router using CPU +GPU:
http://shader.kaist.edu/packetshader/

How far are we with Mikrotik from all these?

-Thanks

Just 1 step behind Linux Kernel i suppose.

So as soon as stable, you can start to drop feature requests.

I personally would prefer to get stably in 10Gbps first

10 G is already stable in linux I think!

It will be interesting to see how it's put into production, right now it's all research with some PoC code. It'll be a while before it's kernel space but user space shouldn't be too far away. I wouldn't expect to see MT putting out RB's with the hardware needed tho, Most of these are referring to way of passing the load from CPU to GPU and to integrate would mean talking to Nvidia to manufacture daughter boards. I think first we would see support for it in x86 systems where you can load up from GPU's

The speedup comes from GPU's parallel nature aswell so you'll need many streams of data to reach these levels of throughput

10gbit card's are stable, You wont hit that throughput tho

Whats wrong in adopting GPU based technology by Mikrotik?

Nothing, except it requires GPU's, unless MT is making them in big numbers Nvidia aren't going to give them raw chips which means they need a normal card and PCIe slot and PSU connectors etc, Mean's a beefy PSU for it aswell, I honestly don't know if the systems needed would support PowerPc either

All in all alot of work, MT could put out a nice 1u x86 system tho....

If you are passing that amount of traffic, you wouldnt really care about Linux kernel or RouterOs. You would most probably be rich enough to afford high end Ciscos

Thats been my point every time someone brings this up, when your pushing 10gbit+ of traffic you want to be able to call someone up when it goes wrong and yell at them till it's fixed, you also expect updates to be tested in the lab.

Support is not the real problem with software routers.


I think the main problem is power consumption. Getting 10, 40 or 100 Gbps routing without packet loss on X86 even with GPU is very power hungry.

It is something for geeks, not something we can put in a Datacenter for real use.


X86 and GPUs are good for desktop computing and technical / scientific workstations. Not more.


Routing with X86 and GPUs is like using a Boeing 747 to go to the supermarket.


Try to do a Cisco 7600 routing task (1 terabit/sec) with X86 machines. You will end needing ten times more power and five times more room.

This is not terribly attractive, except if you are in love for X86 machines and you can't immagine to stop giving money to Intel or AMD.

RouterBoards are using Level 2 logic circuits, giving basic level 2 switching at wire speed without packet loss (i can confirm this as i tested it with a professional Ethernet Tester at 1 Gbps full duplex).

The next step to get efficient and very fast routing with low cost products is to develope level 3 code for logic circuits, using FPGAs or similar circuits. This need porting level 3 Linux kernel routing code to those circuits (HDL language programming, very interesting area).

Those ports should be integrated inside Linux kernel.org, so that a good support, development responsiveness and perenniality could be achieved.

Then you could have routing with 100 Gbps speed at a reasonnable price, and without enormous power consumption. You will not have this before, except perhaps with futur low cost Linksys or similar products, but a with a fraction of the functions you can have on Router OS, and without support at all.

Do not forget that a router do have generally at least two ports. Then if you have two 100 Gbps ports with full duplex streams, then you'll need 200 Gbps of total routing capability, with a total trafic bandwith of 400 Gbps.

If you have a router with ten 100 Gbps ports, then you need a total routing speed of 2 terabits / sec, without packet loss neither jitter. Packet loss and jitter are no more tolerated today in carrier class hardware because of realtime trafic like video and VoIP.

Actually the price of 100 Gbps CFP modules is about 40 000 €. And the warranty only 3 - 6 monthes. So only 10 and 40 Gbps is possible for small and medium providers.

Google for example is using 100 Gbps transceivers and most big providers are using them at least for testing in the lab.

In our country 100 Gbps ports are not really common. Most Ethernet inter providers links are 1 and 10 Gbps, even with tier one providers.

Even for a 10 Gbps ports router, a carrier class router still needs wire speed. If you have said 10 10 Gbps ports, then you need 200 Gbps routing capability, and 400 Gbps total bandwith in the internal data path of the router.

This cannot be supported by a single X86 machine.

Routerboards can do a good work with a few 100 Mbps ports, near wire speed. X86 can do a good job with a few 1 Gbps ports, near wire speed, but i would not use X86 for carrier class routing at 10 or 40 Gbps. You can forgot 100 Gbps even for a single port X86 router.

A tendancy today to lower prices is to try staying level2 in the backbone, using only a mesh of switches. But this need proprietary protocols. I've seen big compatibility problems as soon as you try to extend this architecture, putting down a full provider network very easily.

Provider Backbone suite of Ethernet protocols should allow in the near futur a better interoperability between providers at level 2, allowing circuits like with ATM networks.

RouterBoards are using Level 2 logic circuits, giving basic level 2 switching at wire speed without packet loss (i can confirm this as i tested it with a professional Ethernet Tester at 1 Gbps full duplex).

Have you tested all routerboards or specific ones?

he is talking about RBs with switch ports. Powerful RBs are able to push that 1Gbps through the CPU

You should qualify that, You can push 1gbit if you do nothing but basic routing

yes, I don't find the multi-processor support that effective. I also wonder why mikrotik is taking so long time to release a stable and efficient version with multiprocessor support.

Multi processor is not the right target for a fast router. X86 / X64 and similar processor are not designed to manage small amount of data at very high speed. They have only one bus and their instruction set is not adapted to routing.

Concentrating efforts on logic circuits programming is better. It gives ten times more efficient routers, a lower cost, and ten times less power hungry.

what about performance of vyatta or imagestream on x86 hardware? Are they better? If yes why? And why we can not expect the same with Mikrotik?

Handling 10gbit+ of traffic isn't for the wannabe's. It's a serious amount of traffic and Cisco/Juniper know this. They use ASIC's for a reason. To be honest software routing with this kind of traffic is just plain stupid and with the effort needed to use GPU's with MT they would be better off going down the ASIC/FPGA route.

If you have 10gbit+ of traffic it's not like your not making enough money to buy a decent hardware router

To be able to route efficiently, a FPGA chip needs to be able to make table lockup. I think Cisco is using this function to get route resolution at wire speed.

This give the same routing speed than MPLS, with simple IP routing.

This function is implemented inside high end FPGA devices like Altera Stratix, but i remember there was some low cost chips with this function at Altera.

I did not work on FPGAs since a long time, but simply looking at Altera site show that they have mid range chips like "Arria" with integrated IP hardware functions and low power design. Those chips can be used for 10 - 100 Gbps routing at hardware speed.

I'm not sure they are even needed for mid range 10 - 100 G bandwith routers.
It's certainly possible to design mid range routers with low cost FPGAs, like Cyclone ones from Altera, or equivalent from Xylinx, starting at something like 50 $ - 200 $ each. There are even very low cost chips, starting at 5$ each, with a small amount of gates. I've seen such chips on Routerboards, certainly used for logic glue. Mid range and high end FPGA devices can be very expensive. Some high end FPGA devices are in the 1000 - 3000 € range... This is not possible to use them in low cost products.


What is very nice with FPGAs, is that if you have enough gates available in the chip, you can duplicate the logic modules, and get 2 , 3 or ten time the routing power. Then it's very easy to achieve extremly high bandwith with low power consumption and low development cost.

工厂检验计划tech, I must say you have very good command on hardware design. What is your background?
I hope someday Mikrotik will release a hardware that would handle at least 1 gig ( assuming 40/100gig is not a possibility) very efficiently with connection tracking, filtering and queues all enabled.



Sudipta

Would be good if RouterOs support those NETFCPGA for wirespeed packet processing and forwarding on x86.

http://www.netfpga.org/php/specs.php

Why do you want to use this ?

There is no reason to use this kind of hardware because :

1) you'll need a X86 for almost nothing, except controlling the card.

2) It's certainly expensive.

3) it's PCI based. You buy it, and after a couple years it doesn't work anymore, because you can't find a new compatible motherboard, or driver, or OS.

4) PCI is end of life.


It's better to design low cost embeded devices, like Mikrotik Routerboards ones, but with a mid range FPGA inside to accelerate routing at wire speed like on this board.


如果每个1 Gbps港口dedicated path to the FPGA, then it's possible to have a true multi Gbps wire speed router.


My background was mainly servicing big digital audio mixing desks (Solid State Logic) and digital multitrack recorders (Sony).

I designed as well as a hobby :

- a radio controlled Nixie clock (DCF 77 receiver), with interfacing to a low power real time clock chip, thermal sensor, and own DCF77 decoding code, using an Atmel ATmega 128 controler and C compiler. Status : working and 5 clocks manufactured.
The real time code for the controler and the hardware multiplexer for driving the Nixie tubes have asked quite a lot of work. (about one month if i remember well). It was a very intersting project to learn realtime design with different circuits working together. All clocks are still working after 7 years (without reboot :=). High reliability was a design goal.

- a MADI decoder (MADI is a quite old digital audio format used in the broadcast industry for linking multitrack recoders to mixing desks using 75 ohm coaxial video cables). This project has stayed only theorical, i did use an Altera FPGA development kit.


But i prefer to watch in the foreground :=)

How about Routers based on Intel NPU like the Octeon and Cavium?

I would prefer to stay away from proprietary DSP solutions if i had to do routing designs. History has shown that processor manufacturers can change their strategy and drop support quite fastly (see Intel Itanium...).

Learning DSP programming is not a simple story. This is one reason more to stay away from proprietary DSPs.

I would prefer to have standard VHDL code in my bag, designed from the ground up by my company, so that i could change the FPGA chip brand in case of problem with a chip manufacturer...

VHDL is a programming language used for FPGA logic circuits (VERILOG is often more popular). It is like PALASM for older PAL circuits (Programmable Array Logic) or C++ for serial Cisc or Risc processors.

I've noted that those two solutions are limited to 40 Gbps. FPGAs do allow 100 Gbps and certainly more flexibility.