13

I have a self-written interface tun0 (TUN/TAP based) that outputs what it receives.
I need all traffic of the system to flow through this interface.
The role of the interface is:

  1. To figure out packets likely to be censored and tunnel them.
  2. Pass all other traffic untouched.

As you guess I am trying to build an anticensorship tool.
Decision about tunneling must be taken inside tun0 process
because only there we may use trusted DNS.

I need your help to show me how to make all traffic flow through a self-written interface tun0. If tun0 needs changes I ask you to provide such changes.

Below is how I tried to make all traffic go through tun0 and failed (pings fail).

Compiling

  1. gcc tun0.c
  2. sudo ./a.out

Configuring

  1. sudo ip addr add 10.0.0.1/24 dev tun0
  2. create table John

    $ cat /etc/iproute2/rt_tables 
    #
    # reserved values
    #
    255     local
    254     main
    253     default
    0       unspec
    #
    # local
    #
    #1      inr.ruhep
    
    200 John
    

Order is important:

  1. sudo ip rule add from all lookup John
  2. sudo ip route add default dev tun0 table John
  3. sudo ip rule add iif tun0 lookup main priority 500

    $ ip rule
    0:      from all lookup local 
    500:    from all iif tun0 lookup main 
    32765:  from all lookup John 
    32766:  from all lookup main 
    35000:  from all lookup default 
    

Troubleshooting

  1. sudo tcpdump -i wlp2s0 -qtln icmp and then ping -I tun0 8.8.8.8 shows no packets captured, it means no packets are transmitted from tun0 to wlp2s0 via iif tun0 lookup main rule.

  2. When I replaced tun0 with lo everywhere then it worked for me.

Also Tried

  1. Turning off reverse path filtering, rp_filter=0 in /etc/sysctl.conf

Answer Troubleshooting

iptables -I FORWARD -j LOG --log-prefix "filter/FORWARD " 
iptables -t nat -I OUTPUT -j LOG --log-prefix "nat/OUTPUT " 
iptables -t nat -I PREROUTING -j LOG --log-prefix "nat/PREROUTING " 
iptables -t nat -I POSTROUTING -j LOG --log-prefix "nat/POSTROUTNG "
tail -f /var/log/syslog

Modified sources from answer are also here.

ilyaigpetrov
  • 143
  • 2
  • 11

1 Answers1

10

So in your configuration, all the packets you try to send to the network initially originating from 10.0.0.1 (because they are going through tun0 interface and its local address is 10.0.0.1). You capture the packets, everything is fine so far.
Now, tun0 sends the packets further. Source address is 10.0.0.1 and you want the packets to leave through a different interface (wlp2s0 in your case). That's routing so let's enable routing first:

sysctl -w net.ipv4.ip_forward=1

After that, if you'll look at tcpdump for wlp2s0 you can notice the packets leave with source address 10.0.0.1 and not with the source address of the wlan interface (what you would expect I guess). So we need to change the source address and it's called source NAT. In linux it's easy with help of netfilter/iptables:

iptables -t nat -A POSTROUTING -o wlp2s0 -s 10.0.0.1 -j MASQUERADE

Please also check that your FORWARD chain has ACCEPT policy or you would need to allow forwarding with something like:

iptables -A FORWARD -i tun0 -o wlp2s0 -s 10.0.0.1 -j ACCEPT
iptables -A FORWARD -i wlp2s0 -o tun0 -d 10.0.0.1 -j ACCEPT

Everything should work now: linux kernel does the routing, it's moving packets from tun0 interface to wlp2s0. netfilter should change the source IP 10.0.0.1 to your wlp2s0 interface assigned address for output packets. It memorizes all the connections and when the reply packets go back (if they) it changes the destination address of the wlp2s0 interface assigned address to 10.0.0.1 (the "conntrack" feature).
Well, it should but it doesn't. It seems, netfilter gets confused with this complicated routing configuration and the fact that the same packet first goes through the OUTPUT chain and then being routed and comes to PREROUTING chain. At least on by Debian 8 box it doesn't work.
The best way to troubleshoot netfilter is the TRACE feature:

modprobe ipt_LOG
iptables -t raw -A OUTPUT -p icmp -j TRACE
iptables -t raw -A PREROUTING -p icmp -j TRACE

I only enable tracing for ICMP packets, you may use other filter to debug.
It will show what tables and chains the packet goes through. And I can see that the packet goes no further the FORWARD chain (and it's not being caught by the nat/POSTROUTING chain that actually does SNAT).
Below are several approaches to make this work.

APPROACH #1

The best way to un-confuse netfilter is to change the source IP address of packets in tun0.c application. It's also the most natural way. We need to change 10.0.0.1 to 10.0.0.2 on the way outwards and 10.0.0.2 to 10.0.0.1 on the way back.
I've modified tun0.c with source address change code. Here is the new file and here is patchfile for your tun0.c. Changes to IP header also involve checksum correction, so I took some code from OpenVPN project. Here is the full list of commands I execute after a clean reboot and tun0_changeip.c launch:

ifconfig tun0 inet 10.0.0.1/30 up
sysctl -w net.ipv4.ip_forward=1
ip route add default dev tun0 table John
ip rule add from all lookup John
ip rule add from 10.0.0.2 lookup main priority 500
iptables -t nat -A POSTROUTING -o wlp2s0 -s 10.0.0.2 -j MASQUERADE

Please note that you don't need to turn off the reverse path filtering in that case, because everything is legal - tun0 only receives and sends packets that belong to its subnet. Also you can do a source-based routing instead of interface-based.

APPROACH #2

It's possible to do SNAT before the packet reach tun0 interface. It's not very correct though. You will definitely need to turn off the reverse path filtering in this case:

sysctl -w net.ipv4.conf.tun0.rp_filter=0
# It won't work without also changing the "all" value
sysctl -w net.ipv4.conf.all.rp_filter=0

Now, do SNAT: iptables -t nat -A POSTROUTING -o tun0 -s 10.0.0.1 -j SNAT --to-source ip.address.of.your.wlan.interface

Here we change the source address just before the packets reach the tun0 device. tun0.c code resend these packets "as is" (with changed source address) and they are successfully routed through wlan interface. But you might have a dynamic IP on wlan interface and want to use MASQUERADE (in order to not specify the interface address explicitly). Here is how you can make use of MASQUERADE:

iptables -t nat -A POSTROUTING -o tun0 -s 10.0.0.1 -j SNAT --to-source 10.0.55.1
iptables -t nat -A POSTROUTING -o wlp2s0 -s 10.0.55.1 -j MASQUERADE

Please note the "10.0.55.1" IP address - it's different. You can use any IP here, it doesn't matter. The packets reach nat/POSTROUTING chain on wlp2s0 interface if we change the source IP before. And now it's not dependent on a static IP for wlan interface.

APPROACH #3

You can also use fwmark. That way you don't need SNAT but you'll capture only outgoing packets:
First we need to disable reverse path filtering for tun0 because it will forward packets that belong to another network:

sysctl -w net.ipv4.conf.tun0.rp_filter=0
# It won't work without also changing the "all" value
sysctl -w net.ipv4.conf.all.rp_filter=0

Now let's alter the routing rules a bit:
# Delete old rules
ip rule del iif tun0 lookup main
ip rule del from all lookup John

# Packets will start going from wlan interface so they will have source address of it
iptables -t mangle -A OUTPUT -o wlp2s0 -j MARK --set-mark 1
ip rule add fwmark 0x1 lookup John

That's another "hack" for routing and netfilter that works on my Debian 8 box, but still I recommend to take the first approach as it's more natural and doesn't use any hacks.


You may also consider to build your application as a transparent proxy. I think it would be much easier instead of analyzing packets from tun device.

tifssoft
  • 581
  • 4
  • 8
  • I had to use `-j SNAT`, not `-s SNAT` – ilyaigpetrov Nov 30 '17 at 11:07
  • It works but performance is very intermittent (it may stall for a 10 of seconds then continue working). I'm going to figure why it happens and how to fix this. – ilyaigpetrov Nov 30 '17 at 11:24
  • 1
    Sorry it was my typo. I added another approach to my answer. Have no idea about the performance issue. By the way, why not to use a transparent proxy with iptables DNAT to filter and divert the traffic? – tifssoft Nov 30 '17 at 12:50
  • I'm not able to reproduce your mark approach, I've added only `sudo ip rule add iif tun0 lookup main priority 500` to it but still it did't work. I like this approach, pity that I can't reproduce it. – ilyaigpetrov Nov 30 '17 at 12:59
  • So far from three methods offered I managed to make work only one with MASQUERADE, but it stalls for non-response periods of 10s. – ilyaigpetrov Nov 30 '17 at 13:38
  • Transparent proxy is a very interesting topic, thank you for offering it. – ilyaigpetrov Nov 30 '17 at 13:57
  • Here is complete command list I issued in my clean debian jessie installation to test the fwmark approach: https://pastebin.com/RaYZP25y (I used two virtual terminals, one to run tun0 and another to issue commands). Didn't you forget the rp_filter when you were trying this approach? P.S. you don't need the "ip rule add iif tun0" in this scenario because packets going out from tun0 don't have the fwmark to they will reach main route table anyway. – tifssoft Nov 30 '17 at 14:01
  • Regarding transparent proxy, you can proxy DNS traffic as well, and modify/forward DNS answers as you wish. The DNS protocol is pretty simple, there are a lot of easy implementations in C and other languages. – tifssoft Nov 30 '17 at 14:08
  • I tried your commands from answer and from pastebin verbatim and still couldn't make fwmark work: https://pastebin.com/6jFXSW0s – ilyaigpetrov Nov 30 '17 at 14:46
  • Let us [continue this discussion in chat](http://chat.stackexchange.com/rooms/69508/discussion-between-tifssoft-and-ilyaigpetrov). – tifssoft Nov 30 '17 at 15:09
  • `packets leave with source address 10.0.0.1 and not with the source address of the wlan interface` -- why is this a problem? Doesn't gateway do the NAT and on the way back destination 10.0.0.1 should be handled fine? – ilyaigpetrov Dec 01 '17 at 10:25
  • It's not a problem it's just a requirement to do a SNAT before these packets reach WAN. Normally, iptables -t nat -A POSTROUTING -o wlp2s0 -s 10.0.0.1 -j MASQUERADE does it. However doing NAT is just another job your computer does, so if it's possible to avoid this, it's better to avoid. – tifssoft Dec 01 '17 at 11:18
  • About your new solution -- could you, please, append it to the answer with links to modified source codes. Though I'm mostly engaged with the idea of writing a TCP proxy now, I think what you've done shouldn't be lost in vain. – ilyaigpetrov Dec 03 '17 at 12:50
  • I've finally updated my post. The new solution is the 1st approach. Please tell me if you find any mistakes or things you think need to be explained deeper. And thank you for the bounty, you made me really happy! – tifssoft Dec 04 '17 at 16:12
  • 1
    Thank you for your new approach, I followed it step by step and it worked perfectly. Yet I don't understand why we need to change ips, most importantly it works. In case my plans with TCP proxy fail I will be able to return to your answer. You showed a lot of networking skills here and I don't doubt your skills will be coveted. Good luck! – ilyaigpetrov Dec 04 '17 at 17:01