I have the zt1 instance assigned to a loopback interface as the WAN interface IP is not a publicly routable address, the lo0 IP is. I've also tested an L2TP connection from that loopback and it works fine when the loopback is specified as the src-address.

I see the outbound connections on UDP port 9993 but I'm not seeing anything coming back in.. Is there an issue with ZeroTier and binding to a specific interface? I'm going to throw this in a LAB and see what the hell is going on or if I can reproduce.

The Datacenter we connect to claims nothing is being filtered, only unique thing with them is their handoff with us is unnumbered (/32) and not publicly routable but the /27 they've provided to us is public and routable. No NAT, no dynamic routing just a static default route, very basic setup. It all worked fine on the bench, only change was the WAN IP.. All I can think is there is something wrong with the interface binding and ZeroTier.

The datacenter handoff is a SM fiber from their Cisco to our RB5009 (ROS7.8) with a /32 (unnumbered) assigned.

Normally, you don't need to specify any interface at all, unless you want to force the traffic in some way. But if you force zt to a specific interface that doesn't have internet access or is filtered, it won't work for obvious reasons. Btw, is the node approved by ZeroTIer Central?

Doesn't even show up.. It did on the bench and it was approved, but I removed it while I was troubleshooting.

The fact the peer list/path isn't populating makes me think its not communicating properly to the cloud controller / peer to peer network.

I think its just not binding to the correct IP address. I'm going to open a support ticket.

The interface set on the zerotier instance (zt1) controls where it will look for paths. I suspect the problem is the lo0 isn't getting NAT'ed when zerotier tries to communicate the my.zerotier.com controller.

Any reason you're using a loopback interface, instead of a WAN interface? Otherwise you'll need a your own controller available via the lo0.

The interface set on the zerotier instance (zt1) controls where it will look for paths. I suspect the problem is the lo0 isn't getting NAT'ed when zerotier tries to communicate the my.zerotier.com controller.

Any reason you're using a loopback interface, instead of a WAN interface? Otherwise you'll need a your own controller available via the lo0.

Because the DC providing our WAN connection decided to make the /31 subnet between their router and ours inaccesabile from the outside world. I have a feeling this is causing the issue... unless I can bind the managment traffic and dns lookups to the LAN IP addresses this is going to be an uphill battle... that or I'll have to do a whole lot of mangle / nat fuckery to get it to work.

Unless I misunderstood:
Zerotier is outbound. Has nothing to do with inbound ?

通过尿Zerotier不是形成邻接r to peer network.. There are no paths shown in the /zerotier/peer/pr tables. I'd suspect it's also not resolving DNS for my.zerotier.com because I can't speficy the src-address.

What I was aiming for: why the separate loopback interface ?
Keep it normal.

It was part of the troubleshooting process.. was trying to use an IP not in use by a customer device.

But i just proved my theory.. SRC-NAT'd the traffic from the WAN IP to the loopback and zerotier connected right up... Now I'm going to see if I can get a routable subnet on the WAN side so I don't have to screw with this goofy config.

I think it all related to the WAN IP not being internet accessible. If I could have binded the service to specific IP I don't think this would have been an issue.

An Art lens is a special kind of lens made by Sigma that is mainly used for more artistic photographs. Art lenses are often used to create a vintage or classic look, or to add a special effect to a photograph.what is an art lensin photography

So basically what Amm0 already suggested.

@wildbill442, just a couple of suggestions:
- if your router is the default gateway and is working, configure zt to use any interface. Make sure outbound traffic is not filtered.
- If you're behind nat/cg-nat, expose (is dst-nat) the zt instance port to avoid possible forwarding though external servers. You can check the relay status using the "zerotier-cli peers" command on any node that has zerotier installed.

@wildbill442, just a couple of suggestions:
- if your router is the default gateway and is working, configure zt to use any interface. Make sure outbound traffic is not filtered.
- If you're behind nat/cg-nat, expose (is dst-nat) the zt instance port to avoid possible forwarding though external servers. You can check the relay status using the "zerotier-cli peers" command on any node that has zerotier installed.

It was a NAT issue. WAN IP is not reachable via internet, had to src-nat it to one of the LAN ip's that are publicly routed to get it to connect... All good!

[internet] <---> [datacenter_net] <--- filtered_subnet/31 ---> [5009 w/ zeroteir] <-- public/27 -->

Not your standard config.

Glad to hear it worked out! Regarding the network topology, totally agree it's not a typical setup.

@ wildbill

your problem:

I have the zt1 instance assigned to a loopback interface as the WAN interface IP is not a publicly routable address,

actually has been resolved by your self

see the outbound connections on UDP port 9993 but I'm not seeing anything coming back in.

it was rejected by your own router.

then... you have made it worked

It was a NAT issue. WAN IP is not reachable via internet, had to src-nat it to one of the LAN ip's that are publicly routed to get it to connect... All good!

[internet] <---> [datacenter_net] <--- filtered_subnet/31 ---> [5009 w/ zeroteir] <-- public/27 -->

probably, for many of us, handed over a private non internet routable ip was weird. isp not necessarily doing nat for you. it is resource intensive for their router.

but, from the service provider perspective, that is how they reserve those limited ipv4.

no need to worry, those /27 for your lo0 - their bgp router knows how to reach those ip via that private wan ip interface. they injected a static route for your /27 on their wan side connected to your router.

just do on your side, some dst-nat to the lo0 public ip.

congratulations. well done