Docker + Bridges + DHCP

Question

I have a lot of docker containers that I need addressable on the the same LAN as their hosts. Up until now, I have been achieving this by using setting up a bridge and manually assigning them IPs, and managing the IPs myself. An example startup would be like so:

docker run \
--net="none" \ \
--lxc-conf="lxc.network.type = veth" \
--lxc-conf="lxc.network.ipv4 = 192.168.1.3/24" \
--lxc-conf="lxc.network.ipv4.gateway = 192.168.1.254" \
--lxc-conf="lxc.network.link = br0" \
--lxc-conf="lxc.network.name = eth0" \
--lxc-conf="lxc.network.flags = up" \
-d [Docker Image ID]

With the host having the bridge defined in /etc/network/interfaces (ubuntu) like so:

auto eth0
iface eth0 inet manual

auto br0
iface br0 inet static
        address 192.168.1.2
        netmask 255.255.255.0
        gateway 192.168.1.254
        bridge_ports eth0
        bridge_stp off
        bridge_fd 0
        bridge_maxwait 0

Since I discovered serf, I have been trying to move over to using automatic discovery within the containers, so that DHCP can keep track of IPs and hand them out to containers. I since changed the startup command to:

docker run \
--net="none" \
--lxc-conf="lxc.network.type = veth" \
--lxc-conf="lxc.network.link = br0" \
--lxc-conf="lxc.network.flags = up" \
-d [Docker Image ID]
/bin/bash

and the bridge to:

auto br0
iface br0 inet dhcp
        bridge_ports eth0
        bridge_stp off
        bridge_fd 0
        bridge_maxwait 0

This resulted in the container starting up, but not having an IP. I then took advice from an online post who managed to get it done with Fedora, by calling dhclient. Unfortunately this is not working for me within ubuntu based containers.

Below are the following error messages I get under different conditions:

Running dhclient when I have enabled --privileged in starting the container:

dhclient: error while loading shared libraries: libc.so.6: cannot open shared object file: Permission denied

Running sudo dhclient eth0 when not in --privileged

RTNETLINK answers: Operation not permitted
mv: cannot move '/etc/resolv.conf.dhclient-new.31' to '/etc/resolv.conf': Device or resource busy

Running sudo dhclient or dhclient (no interface specified).

Returns immediately and there is still no IP or network connectivity.

How can I get docker containers to grab dynamic IPs from the same subnet as their hosts, such that I can deploy containers across multiple hosts without tracking IPs?

Extra Info

Running DOCKER_OPTS="-e lxc" in /etc/default/docker
Host is Ubuntu 14.04
Docker containers are built using from ubuntu:14.04 in the Dockerfile.

I am seeking for the very same thing. I have a bridge already set up and connected to it a vm that does all the networking - including dhcp. Where do those `--lxc-conf` flags come from? I cannot find anything in the documentation about them? — philipp, Mar 19 '23 at 04:38

score 3 · Accepted Answer · answered Sep 17 '14 at 08:07

3

It appears that this is an open issue and is this is specific to Ubuntu containers and apparmor.

A workaround was posted in there from bprodoehl:

Start the container as privileged with --privileged
Add the following line to the dockerfile: RUN mv /sbin/dhclient /usr/sbin/dhclient
Run dhclient eth0 and you will still see the error message: mv: cannot move '/etc/resolv.conf.dhclient-new.29' to '/etc/resolv.conf': Device or resource busy, but you will now have an IP and you can use the network.

answered Sep 17 '14 at 08:07

Programster

2,157
9
23
35

it works like a charm for my case, but why is that mv needed? Can you explain please? – Federico Bonelli Oct 28 '14 at 08:50
That was what was in the solution the guy posted (which I linked to) and it appears to work. Perhaps someone else can explain. – Programster Oct 28 '14 at 12:13
1

@Federico Bonelli: Apparmor applies security restrictions to the applications it manages. It manages only applications that it has policy definitions for. These policies match applications based on their path. When you move `dhclient` from `/sbin` to `/usr/sbin` the Apparmor policy definition will not match anymore and Apparmor will not apply any security restrictions to this "unknown" application. – paprika Jan 19 '15 at 09:25
I had to resort to "sudo apparmor_parser -R /etc/apparmor.d/sbin.dhclient" as the workaround didn't, this was an ubuntu 14.04 container fyi. However the error message above was seen and an IP address was then assigned. – Goblinhack Feb 23 '15 at 23:23

score 1 · Answer 2 · edited Jan 09 '15 at 12:03

I've found a script

https://github.com/jkrauska/tech-notes/blob/master/docker-dhclient.md

that does exactly what you want (the workaround mentioned by Programster).

The mv command is needed, because when you run docker container in privileged mode, docker not define AppArmor profile for container. So machine-default AppArmor profile is used, and it prevents you from running dhclient at it's default path.

score 1 · Answer 3 · edited Jan 14 '15 at 15:48

if you are trying to get a dhcp address on an ubuntu docker container just do the following:

set dns option on you docker daemon command (--dns <my_dns_ip>)
open /etc/dhcp/dhclient.conf and edit the line which contains request subnet-mask, broadcast-address... and remove the words domain-name, domain-name-servers
then after applying service networking restart you'll get a new dhcp address without error messages

score -2 · Answer 4 · edited Jul 23 '15 at 15:34

-2

The other possibility is to use "pipework." You need to run it outside of your container. https://github.com/jpetazzo/pipework

edited Jul 23 '15 at 15:34

Community

1

answered Nov 04 '14 at 21:52

user90299

1

5

Hi and welcome to the site. We like answers to be a bit more comprehensive here. Could you edit and explain what `pipework` is, where one can find it, how it can be installed, perhaps give an example of its use? As it stands, your answer is a comment and not an answer. – terdon Nov 04 '14 at 21:54

Docker + Bridges + DHCP

4 Answers4

Linked