Why do you configure wireguard on device B, not device A?
What does the other side look like? Does it have a public address without NAT? If it does: Things should work without destination NAT if connection is initiated from device behind NAT.
Device A has a public IP address (no CGNAT) via its LTE modem. I port forward (DSTNAT) UDP 31232 to Device B which has the WireGuard instance.Ah, I misread and misunderstood some details. So both peers are behind NAT, one is supposed to be reachable via destination NAT.
Never tried that with wireguard, no idea if this should work.
What routerOS version is running on the Mikrotik router doing your port forward?I tried quick test with RouterOS as WG server behind NAT with forwarded port, same as OP has, and it works fine. Exactly as expected, since WG shouldn't care about NAT at all.
[admin@雷竞技网站MikroTik] / ip > nat /防火墙/公关旗帜:X -disabled, I - invalid; D - dynamic 0 ;;; defconf: masquerade chain=srcnat action=masquerade out-interface-list=WAN log=no log-prefix="" ipsec-policy=out,none 1 ;;; ;;; force DNS chain=dstnat action=dst-nat to-addresses=192.168.10.2 protocol=udp src-address=!192.168.10.2 dst-address=!192.168.10.2 in-interface=bridge dst-port=53 log=no log-prefix="" 2 ;;; ;;; force DNS chain=srcnat action=masquerade protocol=udp src-address=192.168.10.0/24 dst-address=192.168.10.2 dst-port=53 log=no log-prefix="" 3 ;;; DSTNAT for Device B WireGuard chain=dstnat action=dst-nat to-addresses=192.168.10.5 to-ports=13232 protocol=udp in-interface=lte1 dst-port=13232 log=no log-prefix="" 4 ;;; DSTNAT for Device B webfig chain=dstnat action=dst-nat to-addresses=192.168.10.5 to-ports=80 protocol=tcp in-interface=lte1 dst-port=8080 log=yes log-prefix="8080_dstnat" [admin@MikroTik] /ip/firewall>
[admin@MikroTik] > ip address/ print Flags: D - DYNAMIC Columns: ADDRESS, NETWORK, INTERFACE # ADDRESS NETWORK INTERFACE ;;; defconf 0 192.168.10.1/24 192.168.10.0 ether1 1 D 123.209.117.65/32 123.209.117.65 lte1 [admin@MikroTik] >
[admin@MikroTik] > ip route/ pr Flags: D - DYNAMIC; A - ACTIVE; C - CONNECT, m - MODEM Columns: DST-ADDRESS, GATEWAY, DISTANCE DST-ADDRESS GATEWAY D DAm 0.0.0.0/0 lte1 2 DAC 123.209.117.65 lte1 0 DAC 192.168.10.0/24 bridge 0 [admin@MikroTik] >
[admin@MikroTik] > /ip fire ex # sep/09/2020 20:32:55 by RouterOS 7.1beta2 # software id = 8DD5-P647 # # model = RBD53G-5HacD2HnD # serial number = C8CA0CB0B626 /ip firewall address-list add address=192.168.10.11-192.168.10.255 list=lan_clients add address=192.168.10.100 list=support /ip firewall filter add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid log=yes log-prefix=drop add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp add action=accept chain=input comment="defconf: accept to local loopback (for CAPsMAN)" dst-address=127.0.0.1 add action=drop chain=input comment="defconf: drop all not coming from LAN" in-interface-list=!LAN log=yes log-prefix=drop add action=accept chain=forward comment="defconf: accept in ipsec policy" disabled=yes ipsec-policy=in,ipsec add action=accept chain=forward comment="defconf: accept out ipsec policy" disabled=yes ipsec-policy=out,ipsec add action=fasttrack-connection chain=forward comment="defconf: fasttrack" connection-state=established,related add action=accept chain=forward comment="defconf: accept established,related, untracked" connection-state=established,related,untracked add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid add action=drop chain=forward comment="defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat connection-state=new in-interface-list=WAN /ip firewall nat add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface-list=WAN add action=dst-nat chain=dstnat comment=";;; force DNS" dst-address=!192.168.10.2 dst-port=53 in-interface=bridge protocol=udp src-address=!192.168.10.2 to-addresses=192.168.10.2 add action=masquerade chain=srcnat comment=";;; force DNS" dst-address=192.168.10.2 dst-port=53 protocol=udp src-address=192.168.10.0/24 add action=dst-nat chain=dstnat comment="DSTNAT for Device B WireGuard" dst-port=13232 in-interface=lte1 protocol=udp to-addresses=192.168.10.5 to-ports=13232 add action=dst-nat chain=dstnat comment="DSTNAT for Device B webfig" dst-port=8080 in-interface=lte1 log=yes log-prefix=8080_dstnat protocol=tcp to-addresses=192.168.10.5 to-ports=80 [admin@MikroTik] >
[admin@MikroTik] > ip address/ print Columns: ADDRESS, NETWORK, INTERFACE # ADDRESS NETWORK INTERFACE 0 192.168.10.5/24 192.168.10.0 ether1 1 192.168.201.1/24 192.168.201.0 wireguard1 [admin@MikroTik] >
[admin@MikroTik] > ip route/ pr Flags: D - DYNAMIC; A - ACTIVE; C - CONNECT, S - STATIC, m - MODEM Columns: DST-ADDRESS, GATEWAY, DISTANCE # DST-ADDRESS GATEWAY D 0 AS 0.0.0.0/0 192.168.10.1 1 DAC 192.168.10.0/24 bridge 0 DAC 192.168.201.0/24 wireguard1 0 [admin@MikroTik] >
[admin@MikroTik] > /ip fire ex # sep/09/2020 20:28:42 by RouterOS 7.1beta2 # software id = 50RA-6BBJ # # model = RBcAPGi-5acD2nD # serial number = BECD0C73D111 [admin@MikroTik] >
[admin@MikroTik] /interface/wireguard> export # sep/09/2020 20:41:56 by RouterOS 7.1beta2 # software id = 50RA-6BBJ # # model = RBcAPGi-5acD2nD # serial number = BECD0C73D111 /interface wireguard add listen-port=13232 mtu=1420 name=wireguard1 private-key="xxxxxxxxxxxxxxxxxxxxxxxxxxxxx" /interface wireguard peers add allowed-address=192.168.201.10/32 endpoint=100.103.197.44:61497 interface=wireguard1 persistent-keepalive=25 preshared-key="xxxxxxxxxxxxxxxxxxxxxxxxxxxxx" public-key=\ "meLYM9lu4ViXIOkwB8qCre452hBwV9asGJ/2DIzuiCQ=" [admin@MikroTik] /interface/wireguard>
As expected it behaves exactly the same. Packets were always getting through the DSTNAT to the WireGuard instance on Device B. Something happens with the replies to the client attempting connection.Have you tried to access Wireguard on device A via port 8080 and set up DSTNAT to device B to the Wireguard port. Some ISPs block various ports. If port 8080 (TCP) works, it should also work on WG.
SRCNAT / masquerade rule is above the DSTNAT rule for Device B WireGuard instance:Is there are firewall rule that does source NAT just after destination NAT for the incoming packet? Possibly that confuses wireguard...
What interfaces are in interface list "WAN"?
/ip firewall nat add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none \ out-interface-list=WAN add action=dst-nat chain=dstnat comment=";;; force DNS" disabled=yes dst-address=!192.168.10.2 \ dst-port=53 in-interface=bridge protocol=udp src-address=!192.168.10.2 to-addresses=192.168.10.2 add action=masquerade chain=srcnat comment=";;; force DNS" disabled=yes dst-address=192.168.10.2 \ dst-port=53 protocol=udp src-address=192.168.10.0/24 add action=dst-nat chain=dstnat comment="DSTNAT for Device B WireGuard" dst-port=13232 in-interface=\ lte1 protocol=udp to-addresses=192.168.10.5 to-ports=13232 add action=dst-nat chain=dstnat comment="DSTNAT for Device B webfig" disabled=yes dst-port=8080 \ in-interface=lte1 log=yes log-prefix=8080_dstnat protocol=tcp to-addresses=192.168.10.5 \ to-ports=13232 [admin@MikroTik] >
/interface list member add comment=defconf interface=bridge list=LAN add comment=defconf interface=lte1 list=WAN [admin@MikroTik] >
My device A has 6.45.8, but it has nothing to do with it, you can use literally any RouterOS version. Device A doesn't do anything related to WG, it just forwards one udp port to device B and doesn't care what's inside those packets.What routerOS version is running on the Mikrotik router doing your port forward?
That was a configuration typo during testing! I have fixed now and it behaves the same. I am away from home tonight so I can get a different client set up and perform a Wireshark capture.But there's one thing I noticed now, you have 192.168.201.0/24 as address on wireguard1 interface. But it's not correct address, with .0 at the end it's subnet address. Change it to something else, e.g. .1.
[admin@MikroTik] > ip add print Columns: ADDRESS, NETWORK, INTERFACE # ADDRESS NETWORK INTERFACE 0 192.168.10.5/24 192.168.10.0 ether1 1 192.168.201.1/24 192.168.201.0 wireguard1 [admin@MikroTik] >
I established a separate WireGuard connection to Device A from my laptop in the hotel, that Device A WireGuard subnet is 192.168.200.0/24 and my laptop has IP address 192.168.200.11 once connected. Device B has a static route for the 192.168.200.0/24 subnet via 192.168.10.1. I used the packet sniffer streaming feature on Device B (192.168.10.5) to stream the UDP packets back to my laptop in the hotel (192.168.200.11:37008 via WireGuard). So whilst the WireGuard VPN to Device A was established (to allow me to stream the UDP packets back from Devcie B) I then established the WireGuard connection from the laptop to Device B (using command line WireGuard client).And how exactly did you capture this? It looks like strange mix. Some packets are wrapped in TZSP, from 192.168.10.5 to 192.168.200.11 (what's that?), while others look like direct capture from interface. But if it's captured using packet sniffer on 192.168.10.5, as TZSP suggests, then it shouldn't see hotel's 172.20.1.195, but it's in there.
Also, how does client's config look like? I don't think you posted it before.
[Interface] PrivateKey = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx Address = 192.168.201.200/32 #DNS = 192.168.10.2 [Peer] PublicKey = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx #AllowedIPs = 192.168.10.0/24 AllowedIPs = 192.168.201.0/24 Endpoint = libertyrd.duckdns.org:13232 PresharedKey = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx PersistentKeepalive = 25
[Interface] PrivateKey = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx Address = 192.168.201.200/32 #DNS = 192.168.10.2 [Peer] PublicKey = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx #AllowedIPs = 192.168.10.0/24 AllowedIPs = 192.168.201.0/24 Endpoint = libertyrd.duckdns.org:13232 PresharedKey = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx PersistentKeepalive = 25
I have captured the packets again.
The obvious problem is that it contains only initial requests, there's not a single response. It was the same in previous combined capture as well. All non-TZSP packets are from client to server. But if you look at TZSP packets, you can see that device B is sending responses.
So one more thing you can try is to capture packets on device A's WAN interface. If you see responses there, the problem is somewhere else. If not, then they are eaten either by device A, or by something between device A and B (you could try another capture on device A's LAN interface to tell which it is).
The last two sets of packet captures where whilst connecting to Device B only. One WireGuard connection only.I'm running out of ideas. Last one for now, although there's no reason why it should be the problem, did you try to connect only to device B and no other server at the same time?
Not that it would be completely impossible, but dstnat is simple thing, it's there for years, everyone uses it, ... it's not likely that it would get broken and not noticed by many other people. On the other hand, it is beta version, so maybe the chance is slightly higher.
You can test another service. Not tcp, that may be different, but udp. In v7 there's udp support for OpenVPN, so you can test it with that.
What exactly did you test??I didn't expect that it would change anything, but I tested device A with 7.1beta2 and no problem at all, everything works.
The 'broken port forwarding' theory. So no change for device B, but I also used 7.1beta2 on device A.
So how do I troubleshoot my failed installation then? It works if I put a cheap router in front and port forward, it fails when I put a $365 Mikrotik Chateau LTE12 in front and dstnat the WireGuard traffic.Exactly.
On Device B?Does it make a difference if you lower the mtu size on wireguard interfaces?
Yes, on device B and on your client. I think the mtu should match on both sides. No idea what happens if it does not.On Device B?Does it make a difference if you lower the mtu size on wireguard interfaces?
I'm out of ideas.
- TP-Link with LTE works => LTE is ok
- Chateau with ethernet works => Chateau is ok
- Chateau with LTE does not work => ???, but why, when both should be ok?
It would be interesting if someone else could test it with their Chateau and LTE, but so far there isn't anyone else. I assume you did test it with default config with no modifications other than the one dstnat rule, to completely rule out any mistake you could have done, right? Then all what's left is contacting MikroTik support, but it will be "interesting" to explain it, because it would be really weird and hard to believe bug.
Yes, on device B and on your client. I think the mtu should match on both sides. No idea what happens if it does not.
I tested the bug on beta3 and it’s still therewe have possible fix for this issue, that will be included in upcoming version.
Where did you get beta 3 ?I tested the bug on beta3 and it’s still therewe have possible fix for this issue, that will be included in upcoming version.
我的问题都在RouterOS7,验证l雷竞技RouterOS7 Beta 3I thought chateau was only ros v7?
yeah it worksAh, I misread and misunderstood some details. So both peers are behind NAT, one is supposed to be reachable via destination NAT.
Never tried that with wireguard, no idea if this should work.
using a different LTE-router worked without any problems (but didn't allow me to sniff packets going out). After some days and some traces I realized that the raspi sends out the packets with a dscp-mark of AF-41. I removed the mark at the source withsmartphone <--> internet <--> LTE with chateau12 <-->RP962UiGS-5Hac.. <--> raspi with wireguard on port 51820
and immediately after that command the connection started to work. Looks like someone on the internet throws away packets marked with AF-41...iptables -t mangle -A POSTROUTING -p udp --sport 51820 -j DSCP --set-dscp 0x0
Give it a try and see if it's the same problem on your side/ip firewall address-list
add address=.sn.mynetname.net list="MY WAN ADDRESS"
/ip firewall nat
add action=dst-nat chain=dstnat dst-address-list="MY WAN ADDRESS" dst-port=51820 protocol=udp \
to-addresses=192.168.1.23 to-ports=51820
/ip firewall mangle
add action=set-priority chain=postrouting log-prefix=dscp0 new-priority=0 passthrough=no protocol=udp \
src-address=192.168.1.23 src-port=51820
Columns: ADDRESS, NETWORK, INTERFACE # ADDRESS NETWORK INTERFACE 0 D 10.0.0.9/24 10.0.0.0 bridge1 1 10.1.0.1/24 10.1.0.0 wireguard1
#jan/04/2021 00:26:55 by RouterOS 7.1beta3 # software id = 1W79-SWFL # # model = RBD52G-5HacD2HnD # serial number = C6140BFD9F14 /interface wireguard add listen-port=51821 mtu=1420 name=WG /interface wireguard peers add allowed-address=10.0.0.2/32 interface=WG public-key=\ "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" add allowed-address=10.0.0.3/32 interface=WG public-key=\ "yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy" [admin@MikroTik] /interface/wireguard>
[admin@MikroTik] /ip/address> print Columns: ADDRESS, NETWORK, INTERFACE # ADDRESS NETWORK INTERFACE 0 192.168.3.31/24 192.168.3.0 bridgeLocal 1 10.0.0.1/24 10.0.0.0 WG [admin@MikroTik] /ip/address>
[admin@MikroTik] /ip/route> print Flags: D - DYNAMIC; A - ACTIVE; c - CONNECT, s - STATIC, y - COPY Columns: DST-ADDRESS, GATEWAY, DISTANCE # DST-ADDRESS GATEWAY D 0 As 0.0.0.0/0 192.168.3.3 1 DAc 10.0.0.0/24 WG 0 DAc 192.168.3.0/24 bridgeLocal 0 [admin@MikroTik] /ip/route>