1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
|
+++
title = "Using CNI"
weight = 1
+++
This tutorial will show you how to set up networking for a gVisor sandbox using
the [Container Networking Interface (CNI)](https://github.com/containernetworking/cni).
## Install CNI Plugins
First you will need to install the CNI plugins. CNI plugins are used to set up
a network namespace that `runsc` can use with the sandbox.
Start by creating the directories for CNI plugin binaries:
```
sudo mkdir -p /opt/cni/bin
```
Download the CNI plugins:
```
wget https://github.com/containernetworking/plugins/releases/download/v0.8.3/cni-plugins-linux-amd64-v0.8.3.tgz
```
Next, unpack the plugins into the CNI binary directory:
```
sudo tar -xvf cni-plugins-linux-amd64-v0.8.3.tgz -C /opt/cni/bin/
```
## Configure CNI Plugins
This section will show you how to configure CNI plugins. This tutorial will use
the "bridge" and "loopback" plugins which will create the necessary bridge and
loopback devices in our network namespace. However, you should be able to use
any CNI compatible plugin to set up networking for gVisor sandboxes.
The bridge plugin configuration specifies the IP address subnet range for IP
addresses that will be assigned to sandboxes as well as the network routing
configuration. This tutorial will assign IP addresses from the `10.22.0.0/16`
range and allow all outbound traffic, however you can modify this configuration
to suit your use case.
Create the bridge and loopback plugin configurations:
```
sudo mkdir -p /etc/cni/net.d
sudo sh -c 'cat > /etc/cni/net.d/10-bridge.conf << EOF
{
"cniVersion": "0.4.0",
"name": "mynet",
"type": "bridge",
"bridge": "cni0",
"isGateway": true,
"ipMasq": true,
"ipam": {
"type": "host-local",
"subnet": "10.22.0.0/16",
"routes": [
{ "dst": "0.0.0.0/0" }
]
}
}
EOF'
sudo sh -c 'cat > /etc/cni/net.d/99-loopback.conf << EOF
{
"cniVersion": "0.4.0",
"name": "lo",
"type": "loopback"
}
EOF'
```
## Create a Network Namespace
For each gVisor sandbox you will create a network namespace and configure it
using CNI. First, create a random network namespace name and then create
the namespace.
The network namespace path will then be `/var/run/netns/${CNI_CONTAINERID}`.
```
export CNI_PATH=/opt/cni/bin
export CNI_CONTAINERID=$(printf '%x%x%x%x' $RANDOM $RANDOM $RANDOM $RANDOM)
export CNI_COMMAND=ADD
export CNI_NETNS=/var/run/netns/${CNI_CONTAINERID}
sudo ip netns add ${CNI_CONTAINERID}
```
Next, run the bridge and loopback plugins to apply the configuration that was
created earlier to the namespace. Each plugin outputs some JSON indicating the
results of executing hte plugin. For example, The bridge plugin's response
includes the IP address assigned to the ethernet device created in the network
namespace. Take note of the IP address for use later.
```
export CNI_IFNAME="eth0"
sudo -E /opt/cni/bin/bridge < /etc/cni/net.d/10-bridge.conf
export CNI_IFNAME="lo"
sudo -E /opt/cni/bin/loopback < /etc/cni/net.d/99-loopback.conf
```
Get the IP address assigned to our sandbox:
```
POD_IP=$(sudo ip netns exec ${CNI_CONTAINERID} ip -4 addr show eth0 | grep -oP '(?<=inet\s)\d+(\.\d+){3}')
```
## Create the OCI Bundle
Now that our network namespace is created and configured, we can create the OCI
bundle for our container. As part of the bundle's `config.json` we will specify
that the container use the network namespace that we created.
The container will run a simple python webserver that we will be able to
connect to via the IP address assigned to it via the bridge CNI plugin.
Create the bundle and root filesystem directories:
```
sudo mkdir -p bundle
cd bundle
sudo mkdir rootfs
sudo docker export $(docker create python) | sudo tar --same-owner -pxf - -C rootfs
sudo mkdir -p rootfs/var/www/html
sudo sh -c 'echo "Hello World!" > rootfs/var/www/html/index.html'
```
Next create the `config.json` specifying the network namespace.
```
sudo /usr/local/bin/runsc spec
sudo sed -i 's;"sh";"python", "-m", "http.server";' config.json
sudo sed -i "s;\"cwd\": \"/\";\"cwd\": \"/var/www/html\";" config.json
sudo sed -i "s;\"type\": \"network\";\"type\": \"network\",\n\t\t\t\t\"path\": \"/var/run/netns/${CNI_CONTAINERID}\";" config.json
```
## Run the Container
Now we can run and connect to the webserver. Run the container in gVisor. Use
the same ID used for the network namespace to be consistent:
```
sudo runsc run -detach ${CNI_CONTAINERID}
```
Connect to the server via the sandbox's IP address:
```
curl http://${POD_IP}:8000/
```
You should see the server returning `Hello World!`.
## Cleanup
After you are finished running the container, you can clean up the network
namespace .
```
sudo runsc kill ${CNI_CONTAINERID}
sudo runsc delete ${CNI_CONTAINERID}
export CNI_COMMAND=DEL
export CNI_IFNAME="lo"
sudo -E /opt/cni/bin/loopback < /etc/cni/net.d/99-loopback.conf
export CNI_IFNAME="eth0"
sudo -E /opt/cni/bin/bridge < /etc/cni/net.d/10-bridge.conf
sudo ip netns delete ${CNI_CONTAINERID}
```
|
