Connecting the Docker container to the external network via VLAN Tag

I am considering a system that utilizes the Docker container for tests such as network communication and route confirmation. It is inefficient to prepare a large number of PCs for testing, so we would like to utilize a lightweight and speedy Docker container.

The image configuration is as follows. Isolate the host NIC with the 802.1Q VLAN tag and hang the Docker container in each VLAN (segment) so that mutual Ping, Traceroute can check the communication and route.

Ultimately, I want to create a network test automation environment by making tasks like the following as tasks.

VLAN Tag network generation
Container activation
Test execution
Result assessment

Reference information
Environment
- Guest OS
Building Environment
Conclusion - Connecting the Docker container to the external network via VLAN Tag
(Additional notes) Check here for Docker-Compose

Reference information

I referred to the following information as elemental technology. Thank you very much.

Docker multihost network knowledge.sakura.ad.jp
Docker with Open vSwitch blog.kamabokonet.com
I want to make it a simple network configuration instead of Open vSwitch.
Docker Docs docs.docker.com
MACVLAN notes
macvlan_ipvlan_driver_notes.md · GitHub
This picture was easy to understand.

Environment

In the standalone environment of vSphere, there is no problem as long as NIC supports VLAN trunk such as Intel Pro. VLAN trunk is selected in the standard vSwitch port group and all VLANs are passed with 802.1Q tags. Also promiscuous mode is enabled.

HyperVisor: VMware ESXi6.5
NIC: Broadcom NetXtreme BCM5722 Gigabit Ethernet
Switch: Cisco Catalyst2960
Router: Juniper vSRX

Guest OS

Install and use Docker with CentOS. There is no problem if Docker works. The procedure for building Docker is omitted.

CentOS Linux release 7.4.1708 (Core)
Docker version 1.13.1, build 774336d/1.13.1
docker-compose version 1.21.0, build 5920eb0

Building Environment

Build an environment like the one in the opening picture.

Create Docker network

Create the Docker network with MACVLAN driver. The part ens192 of parent adapts to the environment.

$ docker network create -d macvlan \
    --subnet=192.168.20.0/24 \
    --gateway=192.168.20.1 \
    -o parent=ens192.20 vlan20

$ docker network create -d macvlan \
    --subnet=192.168.30.0/24 \
    --gateway=192.168.30.1 \
    -o parent=ens192.30 vlan30

The Docker network was created.

$ docker network ls
NETWORK ID          NAME                DRIVER              SCOPE
61b2b0b3706c        bridge              bridge              local
e14ebf55d626        host                host                local
112dc8a40c20        none                null                local
cacbb0f62e7e        vlan20              macvlan             local
554515e7aa1a        vlan30              macvlan             local

$ docker network inspect vlan20
[
    {
        "Name": "vlan20",
        "Id": "cacbb0f62e7e971c34f9a556e153c367d81ca23eb7d22a2fce8ce35920f031d9",
        "Created": "2018-05-01T23:33:01.063894797-04:00",
        "Scope": "local",
        "Driver": "macvlan",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {
                    "Subnet": "192.168.20.0/24",
                    "Gateway": "192.168.20.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Containers": {},
        "Options": {
            "parent": "ens192.20"
        },
        "Labels": {}
    }
]

$ docker network inspect vlan30
[
    {
        "Name": "vlan30",
        "Id": "554515e7aa1ad11d7df178ce3b6244f087cecf7ca19bb3792915f4f9f7f73de9",
        "Created": "2018-05-01T23:33:03.007612901-04:00",
        "Scope": "local",
        "Driver": "macvlan",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {
                    "Subnet": "192.168.30.0/24",
                    "Gateway": "192.168.30.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Containers": {},
        "Options": {
            "parent": "ens192.30"
        },
        "Labels": {}
    }
]

The CentOS network of the Docker host is in the following state. (Snippet)

$ ip a
2: ens192: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:3a:4c:62 brd ff:ff:ff:ff:ff:ff
4: ens192.10@ens192: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000
    link/ether 00:0c:29:3a:4c:62 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.80/24 brd 192.168.1.255 scope global ens192.10
       valid_lft forever preferred_lft forever
5: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN
    link/ether 02:42:a6:4a:5b:fe brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 scope global docker0
       valid_lft forever preferred_lft forever
    inet6 fe80::42:a6ff:fe4a:5bfe/64 scope link
       valid_lft forever preferred_lft forever
19: ens192.20@ens192: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
    link/ether 00:0c:29:3a:4c:62 brd ff:ff:ff:ff:ff:ff
20: ens192.30@ens192: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
    link/ether 00:0c:29:3a:4c:62 brd ff:ff:ff:ff:ff:ff

Create Docker container

Create a container by specifying the created network and IP address. You can use Ping & Traceroute, so use Alpine. WARNING appears, but ignore this time.

$ docker run -it -d --rm \
    --net=vlan20 \
    --ip=192.168.20.201 \
    --name container-vlan20 \
    alpine /bin/sh

WARNING: IPv4 forwarding is disabled. Networking will not work.
4b5dd6ab28bdaa49b8a64221177b942a3754ae058854081318b14f6e79b9e038

$ docker run -it -d --rm \
    --net=vlan30 \
    --ip=192.168.30.201 \
    --name container-vlan30 \
    alpine /bin/sh

WARNING: IPv4 forwarding is disabled. Networking will not work.
c68efffc09fffb01f44054293bb71213db3f7fb7c72f85300771a5aa11cabb64

Container was created.

$ docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
c68efffc09ff        alpine              "/bin/sh"           3 seconds ago       Up 2 seconds                            container-vlan30
4b5dd6ab28bd        alpine              "/bin/sh"           43 seconds ago      Up 43 seconds                           container-vlan20

Check working

Confirm communication from each container.

$ docker exec -it container-vlan20 /bin/sh

/ # ip a
23: eth0@if19: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue state UNKNOWN
    link/ether 02:42:c0:a8:14:c9 brd ff:ff:ff:ff:ff:ff
    inet 192.168.20.201/24 scope global eth0
       valid_lft forever preferred_lft forever

/ # ping 192.168.20.1
PING 192.168.20.1 (192.168.20.1): 56 data bytes
64 bytes from 192.168.20.1: seq=0 ttl=64 time=1.201 ms
64 bytes from 192.168.20.1: seq=1 ttl=64 time=0.295 ms

/ # ping 192.168.30.201
PING 192.168.30.201 (192.168.30.201): 56 data bytes
64 bytes from 192.168.30.201: seq=0 ttl=63 time=23.294 ms
64 bytes from 192.168.30.201: seq=1 ttl=63 time=0.521 ms

/ # traceroute 192.168.30.201
traceroute to 192.168.30.201 (192.168.30.201), 30 hops max, 46 byte packets
 1  192.168.20.1 (192.168.20.1)  0.343 ms  0.157 ms  0.251 ms
 2  192.168.30.201 (192.168.30.201)  0.323 ms  0.294 ms  0.238 ms

Communication and route confirmation also from the opposite side.

$ docker exec -it container-vlan30 /bin/sh

/ # ip a
24: eth0@if20: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue state UNKNOWN
    link/ether 02:42:c0:a8:1e:c9 brd ff:ff:ff:ff:ff:ff
    inet 192.168.30.201/24 scope global eth0
       valid_lft forever preferred_lft forever

/ # ping 192.168.30.1
PING 192.168.30.1 (192.168.30.1): 56 data bytes
64 bytes from 192.168.30.1: seq=0 ttl=64 time=0.281 ms
64 bytes from 192.168.30.1: seq=1 ttl=64 time=0.293 ms

/ # ping 192.168.20.201
PING 192.168.20.201 (192.168.20.201): 56 data bytes
64 bytes from 192.168.20.201: seq=0 ttl=63 time=0.392 ms
64 bytes from 192.168.20.201: seq=1 ttl=63 time=0.444 ms

/ # traceroute 192.168.20.201
traceroute to 192.168.20.201 (192.168.20.201), 30 hops max, 46 byte packets
 1  192.168.30.1 (192.168.30.1)  0.240 ms  0.176 ms  0.303 ms
 2  192.168.20.201 (192.168.20.201)  0.463 ms  0.216 ms  0.306 ms
/ #

SUCCEEDED !! Mutual communication confirmation and route confirmation are done as expected. As can be seen from the result of Traceroute, the host NIC is transparent to the VLAN tag and is routed between VLANs in the external router.

Conclusion - Connecting the Docker container to the external network via VLAN Tag

By Docker's MACVLAN network driver, we were able to connect the Docker container to the 802.1Q VLAN Tag with the external network. Utilizing the Docker container eliminates the need to prepare a large number of PCs for network testing.