For routing specific client traffic through specific interfaces

Desired Config: An openVPN client connection running from a Tomato/OpenWRT/DDWRT router, routing traffic for only some of our LAN clients through the VPN, while all others route through the default ISP connection.

Enter “ip route” (I've always just used the default “route” in the past). “Ip route” allows the creation of multiple routing tables and “rules” to direct the traffic to a specific table.

Details on everything are below, or cut to the chase.

My VPN provider sets a bunch of routes when making the connection, if you let it. If you don't want that you need to set the openVPN directive “route-noexec”. In Tomato, there's an option called “Create NAT on tunnel”, that is checked. Under Advanced settings, my openVPN client settings are like so:

persist-key
persist-tun
tls-client
auth-user-pass /tmp/auth.txt
comp-lzo
verb 1
reneg-sec 0
route-noexec
route-up /tmp/vpnrouteup.sh
down /tmp/vpnroutedown.sh

The contents of /tmp/auth.txt are simply VPN provider username on the top line, password on the second line. The contents of the route-up and route-down scripts are at the bottom of this page.

For those interested, the “Create NAT on tunnel” option executes the /tmp/etc/openvpn/fw/client1-fw.sh script, which looks like:

#!/bin/sh
iptables -I INPUT -i tun11 -j ACCEPT
iptables -I FORWARD -i tun11 -j ACCEPT
iptables -t nat -I POSTROUTING -s 192.168.1.0/255.255.255.0 -o tun11 -j MASQUERADE

From what I've seen there appear to be 3 default tables:

• main
• local
• default

This can be confirmed via ip rule list:

root@wifi1:/tmp/home/root# ip rule list
0:	from all lookup local 
32766:	from all lookup main 
32767:	from all lookup default

Other tables are defined with integer names. To create a new table simply add a route to it with ip route add [route] table #tablename#:

ip route add 192.168.100.100 via 127.0.0.1 dev lo table 100

To see the table, ip route list table 100:

root@wifi1:/tmp/home/root# ip route list table 100
192.168.100.100 via 127.0.0.1 dev lo 

To add a rule to affect traffic and direct it via routes in your new table 100, ip rule add [blah] table 100, and to see the rule in place, ip rule list:

root@wifi1:/tmp/home/root# ip rule add from 9.9.9.9 to 192.168.100.100 dev lo table 100
root@wifi1:/tmp/home/root# ip rule list
0:	from all lookup local 
32765:	from 9.9.9.9 to 192.168.100.100 iif lo lookup 100 
32766:	from all lookup main 
32767:	from all lookup default

That rule (as far as I can tell) tells the kernel that when traffic arrives on interface lo from ip 9.9.9.9 with destination address 192.168.100.100, then the pertinent routes are found in routing table 100. If you want to direct traffic both ways to and from specific ip's, then you must do so explicitly:

root@wifi1:/tmp/home/root# ip rule add to 9.9.9.9 from 192.168.100.100 dev lo table 100
root@wifi1:/tmp/home/root# ip rule list
0:	from all lookup local 
32764:	from 192.168.100.100 to 9.9.9.9 iif lo lookup 100 
32765:	from 9.9.9.9 to 192.168.100.100 iif lo lookup 100 
32766:	from all lookup main 
32767:	from all lookup default 

If you make a mistake and want to delete those rules, ip rule delete pref #rule#:

root@wifi1:/tmp/home/root# ip rule delete pref 32764
root@wifi1:/tmp/home/root# ip rule delete pref 32765
root@wifi1:/tmp/home/root# ip rule list
0:	from all lookup local
32766:	from all lookup main
32767:	from all lookup default

In my case, the routes that get set by my provider look like so:

root@wifi1:/tmp/home/root# ip route show |grep tun11
10.173.1.5 dev tun11 proto kernel scope link src 10.173.1.6
10.173.1.1 via 10.173.1.5 dev tun11
0.0.0.0/1 via 10.173.1.5 dev tun11
128.0.0.0/1 via 10.173.1.5 dev tun11

My TUN ifconfig in that case looks like:

root@wifi1:/tmp/home/root# ifconfig tun11
tun11 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
inet addr:10.173.1.6 P-t-P:10.173.1.5 Mask:255.255.255.255
UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1
RX packets:17428 errors:0 dropped:0 overruns:0 frame:0
TX packets:19874 errors:0 dropped:16 overruns:0 carrier:0
collisions:0 txqueuelen:100
RX bytes:9120474 (8.6 MiB) TX bytes:3519467 (3.3 MiB)

We can see then that our tun11 IP is .6, our gateway (which is “via” in the table) is .5, and then some default routes are set.

To make these routes work for only some of our LAN clients, we need to add them to a new routing table, and then specify rules to make use of them.

In my case, I'm going to route only traffic from 192.168.1.253 through the VPN.

Manually, that would go like this:

ip route add 10.173.1.5 dev tun11 proto kernel scope link src 10.173.1.6
ip route add 0.0.0.0/1 via 10.173.1.5 dev tun11 table 10
ip route add 128.0.0.0/1 via 10.173.1.5 dev tun11 table 10
ip route add 10.173.1.1 via 10.173.1.5 dev tun11 metric 1 table 10
ip rule add from 10.173.1.0/24 table 10
ip rule add to 10.173.1.0/24 table 10
ip rule add from 192.168.1.253 table 10

Now, since I also have devices getting their DNS servers from DHCP, we'll also need to add rules for those internal devices to make use of the ISPs default route to access DNS servers. To do that, the rules simply address traffic to and from the internal client ip to the DNS servers via the “main” table instead of our new table “10”:

ip rule add from 192.168.1.253 to 89.150.129.10 lookup main
ip rule add from 192.168.1.253 to 89.150.129.22 lookup main
ip rule add to 192.168.1.253 from 89.150.129.10 lookup main
ip rule add to 192.168.1.253 from 89.150.129.22 lookup main

Disclaimer: I have never made use of awk before doing this configuration, so this may be the dumbest awk code ever written. It works, so that's good enough for me.

A basic script to set the routes in my case looks like this:

vpndhosts="192.168.1.253"
dnsservers="89.150.129.10 89.150.129.22"
tun_iface=$(ifconfig|grep tun|awk '{ print $1 }')
tun_inet=$(ifconfig $tun_iface|grep P-t-P|awk -F ':' '{ print $2 }'|awk -F ' ' '{ print $1 }')
tun_ptp=$(ifconfig $tun_iface|grep P-t-P|awk -F ':' '{ print $3 }'|awk -F ' ' '{ print $1 }')
tun_gw=$(echo $tun_inet |awk -F '.' '{print $1"."$2"."$3".1"}')
tun_net=$(echo $tun_inet |awk -F '.' '{print $1"."$2"."$3".0/24"}')
ip route add 0.0.0.0/1 via $tun_ptp dev $tun_iface table 10
ip route add 128.0.0.0/1 via $tun_ptp dev $tun_iface table 10
ip route add $tun_gw via $tun_ptp dev $tun_iface metric 1 table 10
ip rule add from $tun_net table 10
ip rule add to $tun_net table 10
for host in $vpndhosts;do ip rule add from $host table 10;done
for host in $vpndhosts;do for server in $dnsservers;do ip rule add from $host to $server lookup main;ip rule add from $server to $host lookup main;done;done

To add other hosts, vpndhosts is a space delimited list of local ip's.

Unfortunately, getting port-forwarding working from my chosen provider is another issue. Optware here I come.

To delete all custom rules and return to default (in my case at least):

for rule in $(ip rule list |grep -v "all lookup"|awk -F ":" '{ print $1 }');do ip rule delete pref $rule;done

So, add the two code sections above to /tmp/vpnrouteup.sh and /tmp/vpnroutedown.sh and chmod them via the Administration→Scripts→Init script:

cat > /tmp/vpnrouteup.sh <<EOF
#!/bin/sh 
vpndhosts="192.168.1.253"
dnsservers="89.150.129.10 89.150.129.22"
tun_iface=\$(ifconfig|grep tun|awk '{ print \$1 }')
tun_inet=\$(ifconfig \$tun_iface|grep P-t-P|awk -F ':' '{ print \$2 }'|awk -F ' ' '{ print \$1 }')
tun_ptp=\$(ifconfig \$tun_iface|grep P-t-P|awk -F ':' '{ print \$3 }'|awk -F ' ' '{ print \$1 }')
tun_gw=\$(echo \$tun_inet |awk -F '.' '{print \$1"."\$2"."\$3".1"}')
tun_net=\$(echo \$tun_inet |awk -F '.' '{print \$1"."\$2"."\$3".0/24"}')
ip route add 0.0.0.0/1 via \$tun_ptp dev \$tun_iface table 10
ip route add 128.0.0.0/1 via \$tun_ptp dev \$tun_iface table 10
ip route add \$tun_gw via \$tun_ptp dev \$tun_iface metric 1 table 10
ip rule add from \$tun_net table 10
ip rule add to \$tun_net table 10
for host in \$vpndhosts;do ip rule add from \$host table 10;done
for host in \$vpndhosts;do for server in \$dnsservers;do ip rule add from \$host to \$server lookup main;ip rule add from \$server to \$host lookup main;done;done
EOF

cat > /tmp/vpnroutedown.sh <<EOF
#!/bin/sh
for rule in \$(ip rule list |grep -v "all lookup"|awk -F ":" '{ print \$1 }');do ip rule delete pref \$rule;done
EOF

chmod 700 /tmp/vpnrouteup.sh
chmod 700 /tmp/vpnroutedown.sh

bzzt.

Here's the link with basic info.

Setup Optware via the very handy tutorial here.

Install curl via ipkg install libcurl.

Add the following to Administration→Scripts→Init:

#Setup port forwarding client-ID
head -n 100 /dev/urandom | md5sum | tr -d " -" > /tmp/pia_client_id

Setup a scheduled task that runs at least once per hour that contains:

pia_client_id=$(cat /tmp/pia_client_id)
pia_user=$(head -1 /tmp/pia.txt)
pia_pw=$(tail -1 /tmp/pia.txt)
tun_inet=$(ifconfig $tun_iface|grep P-t-P|awk -F ':' '{ print $2 }'|awk -F ' ' '{ print $1 }')

curl -d "user=$pia_user&pass=$pia_pw&client_id=$pia_client_id&local_ip=$tun_inet" https://www.privateinternetaccess.com/vpninfo/port_forward_assignment >> /tmp/port_forward.txt

What's missing is the traffic needs to go out and back over the tunnel- ip rule add?

root@wifi1:/tmp/home/root# ip rule add to 75.52.9.107 table 10
root@wifi1:/tmp/home/root# ip rule add from 75.52.9.107 table 10
root@wifi1:/tmp/home/root# ip rule add to 209.222.18.222 table 10
root@wifi1:/tmp/home/root# ip rule add from 209.222.18.222 table 10
root@wifi1:/tmp/home/root# traceroute 209.222.18.222
traceroute to 209.222.18.222 (209.222.18.222), 30 hops max, 38 byte packets
 1  10.127.1.1 (10.127.1.1)  64.245 ms  64.217 ms  64.661 ms
 2  buc-ird-14sw.voxility.net (93.115.85.33)  64.751 ms  64.891 ms  64.757 ms
 3  buc-ird-02c.voxility.net (93.115.87.29)  64.335 ms  64.456 ms  75.212 ms
 4  buc-ird-01gw.voxility.net (109.163.234.53)  70.970 ms  buc-ird-04gw.voxility.net (195.60.77.13)  65.213 ms
root@wifi1:/tmp/home/root# curl -d "user=$pia_user&pass=$pia_pw&client_id=$pia_client_id&local_ip=$tun_inet" https://www.privateinternetaccess.com/vpninfo/port_
forward_assignment 
curl: (35) error:0D0890A1:asn1 encoding routines:ASN1_verify:unknown message digest algorithm

Hmmmmmmmmmm. Apparently this means curl has an SSL issue.

Ran ipkg install wget-ssl, then /opt/bin/wget –post-data=“user=$pia_user&pass=$pia_pw&client_id=$pia_client_id&local_ip=$tun_inet” –no-check-certificate https://ww w.privateinternetaccess.com/vpninfo/port_forward_assignment and got:

2014-02-19 21:13:51 (633 KB/s) - `port_forward_assignment' saved [57]
root@wifi1:/tmp/home/root# more port_forward_assignment 
{"error":"port forwarding not available for this region"}
root@wifi1:/tmp/home/root# 

Looks like somehow my ip rule add's got dropped or aren't working? Indeed a traceroute shows traffic heading out the wrong interface.

Tomorrow…