Can RouterOS support more than 1 SBE card in a router?

IE I need 3X single port cards for testing and I need 2X four port cards for production.

I need a 10Mb pipe on T1 cards rather than T3..

This is for A telco / ISP provider..

Craig

Am I the only person that runs T1 connections ???

Bueler??

I would imagine this would work as it's just an interface with a gateway...right? We use lots of T's, but haven't done any with a card on a Mikrotik router.

What I am seeing is the router will only see one port ( one card)

If I put more cards in the system it "sees" only one.

Resurces shows the additional cards but they do not show up in the interface list.

Craig

Are you putting these cards on a riser card? We had the same problem. We could only put 1 card in the riser. It would load the driver, but wouldn't show up in the interface list. Try putting them in the pci slots and see if they show up.

Dan

This is a 4 slot mother motherboard (intel)

As I said One will come up the others are "seen" in resources but driver does not "load them up"

Craig

Have you contacted mikrotik support with a suppout file?

It is UP !!

3 T1 cards (SBE) in one router.

I have a 4.5Mb pipe over T1 lines !! (3X single port card per router)

The system is running with 2X RouterOS 2.9.1 with 3X single port cards in each. Cross over cables are being used for the test. (I have arangements with the telco client to use their lab to put actual T1 loops in for testing).

The pipe us up and running, all I need to work on now is "redundancy".
If I unplug any of the 3 Xover cables, the link dies untill it is restored (NO additional action nessisary IE reboot etc). I want it to just drop thruput untill the faulty circuit is restored.

I think I just have a setting incorrect somewhare...

The eventual test will be the use if 2X quad port cards to pump 10Mb of data (IP and VOIP) traffic across 8T1 lines for a FRACTION of the cost of a T3 loop..

The idea is like this:

Telco ---->(10Mb enet) --> RouterOS ---->(8X T1 loops only) -->RouterOS ---> (10Mb enet) =!!

Will post updates...

Craig
http://www.pc-routers.com

What did you have to do to get it to recognize the extra cards?

Sam

I got a file ( My guess is it would be concidered beta )
from support. They sent me a new version of the Sync file (for 2.9)

I will do some more testing on it for them and reply back on the results.

So far, it is rock solid. 4.5Mb on bandwidth tests router to router using 3X single port T1 cards. (SBE)

I will have to get ahold of some quad interface cards to play with..,

So far so good !!

Craig

So here is whare I am now...

I have 3 T1 links bonded via EOPI / Bonding (round robin)..

This is what I see so far..

1) If any T1 link goes down, entire bonding group goes with it.
how ever it goes down, IE pulled cable or by disable.
2)如果羡慕e EOIP circuit fails, the link holds. Aggrigate bandwidth drops accordingly.
3) If any T1 link goes down and the EOIP tunnel goes with it then bonded link stays up. Aggrigate bandwidth drops accordingly.


Conclusion..

The bonding protocol is resiliant in relationship to it's imediate underlying protocols only. (if the EOIP protocols drop, the link simply "offloads the traffic.")
EOIP毫无头绪如果底层IP路径失败。IE a down T1.
If EOIP could be informed of a failed pathway, and show itself as disabled or down (still monitoring the underlying pathway for return of service) then bonding via round robin / EOIP tunnels would truely provide aggrigation and redundancy.

Example:

Assumptions:
1) 3 T1s bound together using EOIP / bondingrr.

Case:
1) All T1s up and running. EOIP running on all interfaces.
Result:
Aggrigated traffic. (IE: aprox 4.5Mb)

2) All T1s up and running EOIP running on only 2 interfaces.
Result:
Aggrigated traffic (IE: aprox. 3Mb)

3) 2T1s online, One down, EOIP running on all Ts
Result:
NO taraffic flow !! as bonding protocol fails over the down link.


Sugestions:
1) Enable link state information to passed from the interface to EOIP.
IE a scripting mech or protocol to shut down the EOIP protocols (or mark them as down (while monitoring th underlying pathway for return of service)).

This would be especialy usefull on the SBE interface devices.

IE if the SBE interface went down or a cable failed or the telco choped the line again...( Qwest !!)then the interface driver would inform all EOIP pipes that the underlying link is down and to go off line and monitor for return of service at which time EOIP would "come back online".

Another option would be the ability link status of an SBE card (or similar) to provide status to the OS. The status could then be monitored and action to control the EOIP tunnel could be monitored by script or similar action. Thus again providing a robust aggrigation solution with resiliancy.

Just my $1.00 worth (after all my hands hurt after that.. I want more than 2bits...)

Craig

Just sent support a copy of the last post..

I will advise as to what they think...

Craig

Latest Update....

It is working !!!

Three bound T1s with load balancing and failover !!

The link is now resiliant even to an unpluged interface.

I belive the "fix" for detecting a down interface wat turning ARP on on all tunnels (EOIP / Bonding).

Craig

Have been testing this and it seems to be working ok but I'm not sure how to check if any of the EoIP interfaces are down and removed from the Bonding interface. Is there anything that indicates that an EoIP interface is down?

If it is not working (not detecting the down tunnel) the entire bonding set will go down. (so if you unplug a line and it stays up then it works...)

You can see the "down" segmant by looking at the traffic across the T1 and the EOIP tunnel for that T. (They will both be 0) I name my T1s and EOIP tunnesl to make it easy to pair them IE T1-XO1 and EOIP-XO1, etc.
(mekes it easyer to tell what is going on..)

希望这可以帮助. .

Craig

Surely if one tunnel goes down but isn't removed from the bundle it would just cause packet loss for every packet sent to the down tunnel (when using rr)?

Apart from checking the traffic flow over the individual EoIP's there isn't any other way?

On loss of one link the tunnel colapses, I mean 0 bps. then it "re-balances" its self and continues to forward packest..

IE 3 T1s = 4.5Mb

One goes down, = 0bps for aprox 1/2 sec then resumes at 3Mb

Craig

The thing I'm looking for is some form of trigger that we can incorporate into our monitoring tools to let us know when one of the EoIP tunnels has dropped. If we don't have that then we will have to periodically log in and check manually which is not a scalable solution.

When the link comes back up the bonding tunnel colapses to 0 then reestablishes with the recovered links' bandwidth added.

IE:

3X T1 with 1 line down... rate = 3.0Mb

When the 3rd line comes up, the tunell drops to 0 then comes back at 4.5 Mb.

As far as monitoring, you could write a script to watch the T1s and the EOIP links end send a message when they were down I think this would be a good one for DUDE..

Craig

As far as monitoring, you could write a script to watch the T1s and the EOIP links end send a message when they were down

Our EoIP links aren't running over T1's and don't change to a down state. The ends of the EoIP links are at each end of a few wireless backhaul links. If I have no way to automate seeing a link go down then bonding is no good for us.

I was having the same issues with the T1s.
I set ARP to enabled on all interfaces / tunnels, and "state detection" started working IE: the "bondset" detected the down link and adjusted.

What ARP has to do with it or why it seems to work, I don't know but it does..

You might want to test this (in a development env.) and see if it gets you what you want.

Craig

Hi Craig,

"arp=enabled" has always been set on the bonding interface, all 3 EoIP tunnel interfaces and the ethernet port that all 3 tunnels run across but still all interfaces remain active even though the far end of 2 of the tunnels don't physically exist at present. If I disable the far end of the third EoIP tunnel traffic stops but still no indication that the tunnel is down.

Things that make you go Hmmmmm.

I wonder why the bonding set can "see" the Ts go down but not the Ethernet....

Oh,,, Ethernet,,, Is there a device IE a DSL modem directly connected to the ethernet port (or hub / swicth) and is the link dropping after this device? IE the DSL line dropping??

If so, then the router does not know that one of its internal interfaces has droped, (as it still has link with a device) in this case you would have to monitor the "link" that EOIP is running over via netwatch or some sort of script..

这是想法:监测链接的远端(both ends looking at the other's endpoint) that EOIP is running over. when a ping fails, disable the EOIP protocol for that link. bonding will colapse for about 1/2 sec. then come back up minus the down EOIP tunnel's bandwidth. when the underlying link comes back online, use an enable statement to re-enable the EOIP tunnel and bonding will colapse for 1/2 sec and come back up "plus" the re-enabled EOIP bandwidth.

I am not sure why EOIP is nessisary for links via ethernet devices,, as I am using EOIP over the SBE links emulate ethernet devices over the serial t1 links so that bonding will think they are ethernet devices....


One other question,, Are ther MikroTik routers at each end?
This one may be a question for Arnis or Normis..

Oh,,, Ethernet,,, Is there a device IE a DSL modem directly connected to the ethernet port (or hub / swicth) and is the link dropping after this device? IE the DSL line dropping??

The EoIP links are running over multiple ethernet and wireless hops obviously with Mikrotik's at each end. When there is a break in the path of one of the tunnels the bonding seems to function correctly but there is just no indication that one of the tunnels has dropped.

这是想法:监测链接的远端(both ends looking at the other's endpoint) that EOIP is running over. when a ping fails, disable the EOIP protocol for that link. bonding will colapse for about 1/2 sec. then come back up minus the down EOIP tunnel's bandwidth. when the underlying link comes back online, use an enable statement to re-enable the EOIP tunnel and bonding will colapse for 1/2 sec and come back up "plus" the re-enabled EOIP bandwidth.

This sounds like the best solution. I was thinking of having the monitoring tools monitor the ip address at the far end but this doesn't really emulate the Mikrotiks fully.

I am not sure why EOIP is nessisary for links via ethernet devices,, as I am using EOIP over the SBE links emulate ethernet devices over the serial t1 links so that bonding will think they are ethernet devices....

Without EoIP how else could I bond 2 multi-hop wireless backhauls together

Yep..
I forgot about the other devices.. I was thinking of "direct connected" routers...



Craig