2

Using wg-quick with the Table=off setting, I managed to set up a non-default interface wg1 on on linux bookworm machine that works as I expect it to:

  • curl ifconfig.co return my real IP.
  • curl --interface wg1 ifconfig.co returns my wireguard server's IP.

I am trying to figure out I can now run a rootless podman container that uses wg1 to communicate with the outside world. By default, podman container use my eth0 interface; podman run -it docker.io/curlimages/curl:latest ifconfig.co returns my real IP, which is what I want. But I cannot find how to do same thing as I did on the host, inside a container.

I thought I could get away with podman run --network=slirp4netns:outbound_addr=wg1 -it docker.io/curlimages/curl:latest ifconfig.co but this returns curl: (6) Could not resolve host: ifconfig.co. It does not seem to be (just?) a DNS issue because using the IP of ifconfig.co instead does not work any better: curl: (7) Failed to connect to 172.64.110.32 port 80 after 75840 ms: Couldn't connect to server.

I guess I have a to set up a podman network myself, or a pod, or something like that, but got lost in the many tutorials I found and never managed to get to the result I wanted: curling through my wireguard server from inside the rootless container.

Thanks in advance for your help.

nicoco
  • 419
  • 1
  • 5
  • 10
  • Using (a convoluted) strace on slirp4netns, I can see that contrary to curl which does setsockopt(... , SO_BINDTODEVICE, ...) slirp4netns just binds to the IP on the interface. That means proper routing has to be in place and that it will affect everything. – A.B Jul 26 '23 at 22:30
  • If for example your local WireGuard address is 10.5.0.2 , I guess you'd get a similar result (timeout) by doing `curl --interface 10.5.0.2 172.64.110.32` (which would thus bind to the address instead of the interface just like slirp4netns). I guess slirp4netns could and should be modified to actually bind to the interface rather than to the address on the interface. – A.B Jul 26 '23 at 22:44
  • Thanks for your comments. Despite my limited understand of networking stuff, I think I arrived to the same conclusion. So there isn't a simple way to get this to work as I intend? There are many tutorials out there, but they all seem awfully complex for what I need. Do you have some pointers maybe? – nicoco Jul 27 '23 at 07:35
  • You need root access and not use slirp4netns network emulation. An other tool accessible to normal users requiring root to grant rights (to use actual network through a privileged helper) is provided with LXC. I don't know how this can be integrated anywhere else. See: https://man7.org/linux/man-pages/man1/lxc-user-nic.1.html – A.B Jul 27 '23 at 07:41
  • I have root access, rootless sounds better but if it's much easier to achieve with the container running as root it's not that much of an issue to me. This is nothing critical, just my homelab. – nicoco Jul 27 '23 at 07:55
  • My bad what I intended works with uid, not gid so I'd rather not embark in anything complex. Anyway with what you said, you should just us podman with its default mode: bridge. https://docs.podman.io/en/latest/markdown/podman-network-create.1.html#example and see what you can do with this (without WireGuard) so one can ponder what can be done later using WireGuard. – A.B Jul 27 '23 at 08:14
  • But the default mode will go via eth0 to communicate with the internet? I'm lost between what route, gateway, etc. I should set. I confess this is all a bit blurry to me :-( – nicoco Jul 27 '23 at 15:58
  • Here's a Q/A as example to do this: https://unix.stackexchange.com/questions/716802/how-to-route-all-traffic-from-a-specific-interface-to-a-specific-gateway-instead . Note that in that Q/A the host will be a router. With slirp4netns network emulation the host is not routing (it emulates routing with host sockets) and this linked Q/A can't apply. With normal networking, the host can be a router. – A.B Jul 27 '23 at 17:11

1 Answers1

2

Background

slirp4netns is a network emulator that uses sockets instead of routing: from the container's point of view, routing is done by its gateway, but actually the host isn't routing: slirp4netns just opens TCP and UDP sockets on the fly for each flow detected coming from the container to handle outgoing traffic.

Using strace on slirp4netns, one can see that currently it doesn't do what curl is doing: it doesn't bind to the interface using setsockopt(..., SO_BINDTODEVICE, ...) but instead binds the socket to the address on the interface. So the route isn't forced to use wg1 (SO_BINDTODEVICE will also force the use of the interface when there's no route left as if it had added a default route on it. Works without issue for layer 3 interfaces like WireGuard, a bit more problematic for layer 2 interfaces without default route with proper gateway on them) and this fails because there's no route to tell to use wg1.

Wrapper

An LD_PRELOAD wrapper can be used to work around the lack of the functionality in libslirp. Altering libslirp's slirp_bind_outbound() behavior would have been the better choice, but I chose instead to alter bind(2): frailer but easier to achieve.

The wrapper checks if a bind attempt (made by slirp4netns) to the IP address (which must be the address on the target interface) provided as environment variable was made, and in such case in addition also binds the socket to the interface using SO_BINDTODEVICE, which is OP's intended behavior, just like the host's curl command is doing. Of course should such bind happen for any other reason than slirp_bind_outbound()'s invocation then expect disruption.

Instructions

Requires a minimal development environment with gcc. Compile with this command:

gcc -o bindtodevicewrapper.so bindtodevicewrapper.c -shared -fPIC -O2 -ldl

the file bindtodevicewrapper.c below:

#define _GNU_SOURCE

#include <dlfcn.h>

#include <sys/types.h>
#include <sys/socket.h>
#include <net/if.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <unistd.h>

#include <string.h>
#include <stdlib.h>

int bind(int sockfd, const struct sockaddr *addr, socklen_t addrlen) {
    static int init=0;
    static int (*orig_bind)(int, const struct sockaddr *, socklen_t);
    static struct in_addr ipv4bindtodevice;
    static struct in6_addr ipv6bindtodevice;
    static char devname[IFNAMSIZ];

    if (!init) {
        if ((orig_bind=dlsym(RTLD_NEXT,"bind")) == NULL) {
            return -1;
        }
        memset(&ipv4bindtodevice, 0xff, sizeof ipv4bindtodevice);   /* invalid for binding */
        memset(&ipv6bindtodevice, 0xff, sizeof ipv6bindtodevice);   /* invalid for binding */
        if (getenv("WRAPPER_BINDTODEVICE")!=NULL) {
            strncpy(devname, getenv("WRAPPER_BINDTODEVICE"), sizeof devname);
            if(getenv("WRAPPER_INET")!=NULL)
                inet_pton(AF_INET, getenv("WRAPPER_INET"),&ipv4bindtodevice);
            if(getenv("WRAPPER_INET6")!=NULL)
                inet_pton(AF_INET6, getenv("WRAPPER_INET6"),&ipv6bindtodevice);
        }
        init=1;
    }

    if (devname != NULL) {
        if (addr->sa_family == AF_INET) {
            if (!memcmp(&((struct sockaddr_in *)addr)->sin_addr, &ipv4bindtodevice, sizeof ipv4bindtodevice))
                setsockopt(sockfd, SOL_SOCKET, SO_BINDTODEVICE, devname, strlen(devname)+1);
        }
        else if (addr->sa_family == AF_INET6) {
            if (!memcmp(&((struct sockaddr_in6 *)addr)->sin6_addr, &ipv6bindtodevice, sizeof ipv6bindtodevice))
                setsockopt(sockfd, SOL_SOCKET, SO_BINDTODEVICE, devname, strlen(devname)+1);
        }
    }

    return orig_bind(sockfd, addr, addrlen);
}

Requires jq to avoid having to guess the IP address on the interface, else one can do it manually for example with WRAPPER_INET=10.5.0.2 if 10.5.0.2 is the address on wg1. This is a single line (with exported variables only for the single command), split with \ for readability:

WRAPPER_BINDTODEVICE=wg1 \
WRAPPER_INET=$(ip -4 -json addr show dev wg1 | jq -r '.[].addr_info[0].local') \
LD_PRELOAD=./bindtodevicewrapper.so \
podman run --network=slirp4netns:outbound_addr=wg1 -it docker.io/curlimages/curl:latest ifconfig.co

Tested working with IPv4 without DNS (not tested with DNS). Should also work with WRAPPER_INET6=... for IPv6 when using slirp4netns's outbound_addr6 but not tested. No privileged access required.

A.B
  • 31,762
  • 2
  • 62
  • 101