I'm trying to write a function that can connect to a server using a specific network interface so that it's consistently routed through that interface's gateway. This is on a macOS system that has one or more VPN connections.
Here's a proof-of-concept test function I've written:
void connectionTest(const char *hostname, int portNumber, const char *interface) {
struct hostent *serverHostname = gethostbyname(hostname);
if (serverHostname == NULL) {
printf("error: no such host\n");
return;
}
int socketDesc = socket(AF_INET, SOCK_STREAM, 0);
int interfaceIndex = if_nametoindex(interface);
if (interfaceIndex == 0) {
printf("Error: no such interface\n");
close(socketDesc);
return;
}
// Set the socket to specifically use the specified interface:
setsockopt(socketDesc, IPPROTO_IP, IP_BOUND_IF, &interfaceIndex, sizeof(interfaceIndex));
struct sockaddr_in servAddr;
bzero((char *)&servAddr, sizeof(servAddr));
servAddr.sin_family = AF_INET;
bcopy((char *)serverHostname->h_addr, (char *)&servAddr.sin_addr.s_addr, serverHostname->h_length);
servAddr.sin_port = htons(portNumber);
if (connect(socketDesc, (struct sockaddr *)&servAddr, sizeof(servAddr)) < 0) {
printf("connect failed, errno: %d", errno);
close(socketDesc);
return;
}
printf("connection succeeded\n");
close(socketDesc);
}
This function will successfully connect so long as the interface is one of the utun interfaces created by the VPNs, or a physical interface that is not used by the VPNs. But if I try to use the physical interface that is used by the VPNs, the function fails with errno 51: Network is unreachable.
For a more specific example, consider a system with the following network interfaces:
en0: Ethernet connection
en1: Wi-Fi connection
utun10: VPN connection 1, connected via en0
utun11: VPN connection 2, also connected via en0
If I call my function with something like:
connectionTest("api.ipify.org", 80, "en1");
connectionTest("api.ipify.org", 80, "utun10");
connectionTest("api.ipify.org", 80, "utun11");
... it will succeed. However, this is what produces the "network unreachable" error:
connectionTest("api.ipify.org", 80, "en0");
Is there some way to have the function work in the case of en0? (Preferably without changing the system's routing table just for this one connection?)
Edit:
It looks like the system doesn't know how to route packets through en0 when the VPN is up, unless it has a non-default route for en0.
I tried using the route command to check which route in the table would be used for a specific interface, and I get the following:
$ route get -ifscope en0 1.1.1.1
route: writing to routing socket: not in table
Only -ifscope en0 produces that error. However, the route table indicates there is a default route for en0. Here is the routing table when only ethernet and the VPN are connected (so no Wi-Fi or second VPN):
$ netstat -rn
Routing tables
Internet:
Destination Gateway Flags Refs Use Netif Expire
0/1 10.16.0.1 UGSc 165 0 utun10
default 192.168.20.1 UGSc 0 0 en0
10.16/16 10.16.0.8 UGSc 3 0 utun10
10.16.0.8 10.16.0.8 UH 2 0 utun10
127 127.0.0.1 UCS 0 0 lo0
127.0.0.1 127.0.0.1 UH 7 7108160 lo0
128.0/1 10.16.0.1 UGSc 40 0 utun10
169.254 link#8 UCS 1 0 en0 !
192.168.20 link#8 UCS 9 0 en0 !
192.168.20.1/32 link#8 UCS 2 0 en0 !
224.0.0/4 link#22 UmCS 0 0 utun10
224.0.0/4 link#8 UmCSI 1 0 en0 !
224.0.0.251 1:0:5e:0:0:fb UHmLWI 0 0 en0
255.255.255.255/32 link#22 UCS 0 0 utun10
255.255.255.255/32 link#8 UCSI 0 0 en0 !
There's clearly a default route listed for en0 pointing to its gateway, 192.168.20.1. Why isn't the packet being routed? If I create a static route for 1.1.1.1/32 or even 1/8 it will work. But so long as en0 only has a default route, it won't work. It's like the default route has been disabled somehow.
Edit 2:
If I add a new route to the table using:
$ route add -ifscope en0 0/0 192.168.20.1
so that the routing table now includes the following entry:
Destination Gateway Flags Refs Use Netif Expire
default 192.168.20.1 UGScI 1 0 en0
alongside all of the above entries, so there are now two default entries, then the connection works. Why is it necessary for there to be an interface-specific default route in order for this to work?
CodePudding user response：
Once you added the routing table to your question, your problem became obvious.
It is the routing table that determines to which gateway a packet is sent. The routing table tells the sending host to which gateway the packet is sent. It does that by comparing the destination address to the routes in the routing table. The most-specific (longest match) route is used. A default route is the least-specific (shortest match) route, and it is used as the route of last resort when there are no more-specific routes in the routing table.
Based on the routing table you provided, any packet with a destination address from 1.0.0.0 to 126.255.255.255 (0.0.0.0/8 and 127.0.0.0/8 are exceptions as unusable ranges) will match the 0/1 routing table entry rather than the default route (0/0), and any packet with a destination address from 128.0.0.0 to 223.255.255.255 (224.0.0.0/4 is multicast, and 240.0.0.0/4 is unusable) will match the 128/1 routing table entry rather than the default route (0/0), because the route length of 1 is more specific than the default route length of 0. That means any packets destined to an address in those ranges (combined, all addresses destined for a different network) will be sent to the gateway (10.16.0.1) referenced by the routing table entries for the 0/1 and 128/1 routes.
To solve your problem, you need to remove the 0/1 and 128/1 routing table entries and replace them with one or more entries that are restricted to the networks which the tunnel can reach. With that, the entries not matching the tunnel route(s) or other more specific routing table entries will use the default route.