diff options
Diffstat (limited to 'g3doc')
22 files changed, 897 insertions, 191 deletions
diff --git a/g3doc/BUILD b/g3doc/BUILD index c315d38be..f91a77b6f 100644 --- a/g3doc/BUILD +++ b/g3doc/BUILD @@ -31,7 +31,7 @@ doc( category = "Project", permalink = "/community/", subcategory = "Community", - weight = "95", + weight = "10", ) doc( @@ -40,5 +40,5 @@ doc( category = "Project", permalink = "/community/style_guide/", subcategory = "Community", - weight = "10", + weight = "99", ) diff --git a/g3doc/README.md b/g3doc/README.md index 304a91493..22bfb15f7 100644 --- a/g3doc/README.md +++ b/g3doc/README.md @@ -117,9 +117,7 @@ for more information on filesystem bundles. `runsc` implements multiple commands that perform various functions such as starting, stopping, listing, and querying the status of containers. -### Sentry - -<a name="sentry"></a> <!-- For deep linking. --> +### Sentry {#sentry} The Sentry is the largest component of gVisor. It can be thought of as a application kernel. The Sentry implements all the kernel functionality needed by @@ -136,9 +134,7 @@ calls it makes. For example, the Sentry is not able to open files directly; file system operations that extend beyond the sandbox (not internal `/proc` files, pipes, etc) are sent to the Gofer, described below. -### Gofer - -<a name="gofer"></a> <!-- For deep linking. --> +### Gofer {#gofer} The Gofer is a standard host process which is started with each container and communicates with the Sentry via the [9P protocol][9p] over a socket or shared @@ -146,13 +142,13 @@ memory channel. The Sentry process is started in a restricted seccomp container without access to file system resources. The Gofer mediates all access to the these resources, providing an additional level of isolation. -### Application +### Application {#application} The application is a normal Linux binary provided to gVisor in an OCI runtime bundle. gVisor aims to provide an environment equivalent to Linux v4.4, so applications should be able to run unmodified. However, gVisor does not presently implement every system call, `/proc` file, or `/sys` file so some -incompatibilities may occur. See [Commpatibility](./user_guide/compatibility.md) +incompatibilities may occur. See [Compatibility](./user_guide/compatibility.md) for more information. [9p]: https://en.wikipedia.org/wiki/9P_(protocol) diff --git a/g3doc/architecture_guide/performance.md b/g3doc/architecture_guide/performance.md index 39dbb0045..b981f0c01 100644 --- a/g3doc/architecture_guide/performance.md +++ b/g3doc/architecture_guide/performance.md @@ -30,7 +30,7 @@ is distinct from **structural costs**. Improvements here are ongoing and driven by the workloads that matter to gVisor users and contributors. This page provides a guide for understanding baseline performance, and calls out -distint **structural costs** and **implementation costs**, highlighting where +distinct **structural costs** and **implementation costs**, highlighting where improvements are possible and not possible. While we include a variety of workloads here, it’s worth emphasizing that gVisor @@ -211,7 +211,7 @@ url="/performance/applications.csv" title="perf.py http.(node|ruby) The above figure shows the result of simple `node` and `ruby` web services that render a template upon receiving a request. Because these synthetic benchmarks -do minimal work per request, must like the `redis` case, they suffer from high +do minimal work per request, much like the `redis` case, they suffer from high overheads. In practice, the more work an application does the smaller the impact of **structural costs** become. diff --git a/g3doc/architecture_guide/resources.md b/g3doc/architecture_guide/resources.md index 1dec37bd1..fc997d40c 100644 --- a/g3doc/architecture_guide/resources.md +++ b/g3doc/architecture_guide/resources.md @@ -19,12 +19,12 @@ sandboxed process: Much like a Virtual Machine (VM), a gVisor sandbox appears as an opaque process on the system. Processes within the sandbox do not manifest as processes on the -host system, and process-level interactions within the sandbox requires entering +host system, and process-level interactions within the sandbox require entering the sandbox (e.g. via a [Docker exec][exec]). ## Networking -The sandbox attaches a network endpoint to the system, but runs it's own network +The sandbox attaches a network endpoint to the system, but runs its own network stack. All network resources, other than packets in flight on the host, exist only inside the sandbox, bound by relevant resource limits. diff --git a/g3doc/architecture_guide/security.md b/g3doc/architecture_guide/security.md index b99b86332..9363d834c 100644 --- a/g3doc/architecture_guide/security.md +++ b/g3doc/architecture_guide/security.md @@ -104,7 +104,7 @@ interactions with a guest operating system and a set of virtualized hardware devices. These hardware devices are then implemented via the host System API by a Virtual Machine Monitor (VMM). The Sentry similarly prevents direct interactions by providing its own implementation of the System API that the -application must interact with. Applications are not able to to directly craft +application must interact with. Applications are not able to directly craft specific arguments or flags for the host System API, or interact directly with host primitives. diff --git a/g3doc/style.md b/g3doc/style.md index d10549fe9..8258b0233 100644 --- a/g3doc/style.md +++ b/g3doc/style.md @@ -46,6 +46,15 @@ protected. Each field or variable protected by a mutex should state as such in a comment on the field or variable declaration. +### Function comments + +Functions with special entry conditions (e.g., a lock must be held) should state +these conditions in a `Preconditions:` comment block. One condition per line; +multiple conditions are specified with a bullet (`*`). + +Functions with notable exit conditions (e.g., a `Done` function must eventually +be called by the caller) can similarly have a `Postconditions:` block. + ### Unused returns Unused returns should be explicitly ignored with underscores. If there is a diff --git a/g3doc/user_guide/FAQ.md b/g3doc/user_guide/FAQ.md index 89df65e99..514fe3918 100644 --- a/g3doc/user_guide/FAQ.md +++ b/g3doc/user_guide/FAQ.md @@ -74,11 +74,10 @@ directories. ### I'm getting an error like: `panic: unable to attach: operation not permitted` or `fork/exec /proc/self/exe: invalid argument: unknown` {#runsc-perms} -Make sure that permissions and the owner is correct on the `runsc` binary. +Make sure that permissions is correct on the `runsc` binary. ```bash -sudo chown root:root /usr/local/bin/runsc -sudo chmod 0755 /usr/local/bin/runsc +sudo chmod a+rx /usr/local/bin/runsc ``` ### I'm getting an error like `mount submount "/etc/hostname": creating mount with source ".../hostname": input/output error: unknown.` {#memlock} @@ -96,6 +95,30 @@ containerd. See [issue #1765](https://gvisor.dev/issue/1765) for more details. +### I'm getting an error like `RuntimeHandler "runsc" not supported` {#runtime-handler} + +This error indicates that the Kubernetes CRI runtime was not set up to handle +`runsc` as a runtime handler. Please ensure that containerd configuration has +been created properly and containerd has been restarted. See the +[containerd quick start](containerd/quick_start.md) for more details. + +If you have ensured that containerd has been set up properly and you used +kubeadm to create your cluster please check if Docker is also installed on that +system. Kubeadm prefers using Docker if both Docker and containerd are +installed. + +Please recreate your cluster and set the `--cni-socket` option on kubeadm +commands. For example: + +```bash +kubeadm init --cni-socket=/var/run/containerd/containerd.sock` ... +``` + +To fix an existing cluster edit the `/var/lib/kubelet/kubeadm-flags.env` file +and set the `--container-runtime` flag to `remote` and set the +`--container-runtime-endpoint` flag to point to the containerd socket. e.g. +`/var/run/containerd/containerd.sock`. + ### My container cannot resolve another container's name when using Docker user defined bridge {#docker-bridge} This is normally indicated by errors like `bad address 'container-name'` when diff --git a/g3doc/user_guide/containerd/BUILD b/g3doc/user_guide/containerd/BUILD new file mode 100644 index 000000000..979d46105 --- /dev/null +++ b/g3doc/user_guide/containerd/BUILD @@ -0,0 +1,33 @@ +load("//website:defs.bzl", "doc") + +package( + default_visibility = ["//website:__pkg__"], + licenses = ["notice"], +) + +doc( + name = "quick_start", + src = "quick_start.md", + category = "User Guide", + permalink = "/docs/user_guide/containerd/quick_start/", + subcategory = "Containerd", + weight = "10", +) + +doc( + name = "configuration", + src = "configuration.md", + category = "User Guide", + permalink = "/docs/user_guide/containerd/configuration/", + subcategory = "Containerd", + weight = "90", +) + +doc( + name = "containerd_11", + src = "containerd_11.md", + category = "User Guide", + permalink = "/docs/user_guide/containerd/containerd_11/", + subcategory = "Containerd", + weight = "99", +) diff --git a/g3doc/user_guide/containerd/configuration.md b/g3doc/user_guide/containerd/configuration.md new file mode 100644 index 000000000..5d485c24b --- /dev/null +++ b/g3doc/user_guide/containerd/configuration.md @@ -0,0 +1,70 @@ +# Containerd Advanced Configuration + +This document describes how to configure runtime options for +`containerd-shim-runsc-v1`. This follows the +[Containerd Quick Start](./quick_start.md) and requires containerd 1.2 or later. + +### Update `/etc/containerd/config.toml` to point to a configuration file for `containerd-shim-runsc-v1`. + +`containerd-shim-runsc-v1` supports a few different configuration options based +on the version of containerd that is used. For versions >= 1.3, it supports a +configurable `ConfigPath` in the containerd runtime configuration. + +```shell +cat <<EOF | sudo tee /etc/containerd/config.toml +disabled_plugins = ["restart"] +[plugins.linux] + shim_debug = true +[plugins.cri.containerd.runtimes.runsc] + runtime_type = "io.containerd.runsc.v1" +[plugins.cri.containerd.runtimes.runsc.options] + TypeUrl = "io.containerd.runsc.v1.options" + # containerd 1.3 only! + ConfigPath = "/etc/containerd/runsc.toml" +EOF +``` + +When you are done restart containerd to pick up the new configuration files. + +```shell +sudo systemctl restart containerd +``` + +### Configure `/etc/containerd/runsc.toml` + +> Note: For containerd 1.2, the config file should named `config.toml` and +> located in the runtime root. By default, this is `/run/containerd/runsc`. + +The set of options that can be configured can be found in +[options.go](https://github.com/google/gvisor/blob/master/pkg/shim/v2/options/options.go). + +#### Example: Enable the KVM platform + +gVisor enables the use of a number of platforms. This example shows how to +configure `containerd-shim-runsc-v1` to use gvisor with the KVM platform. + +Find out more about platform in the +[Platforms Guide](../../architecture_guide/platforms.md). + +```shell +cat <<EOF | sudo tee /etc/containerd/runsc.toml +[runsc_config] +platform = "kvm" +EOF +``` + +### Example: Enable gVisor debug logging + +gVisor debug logging can be enabled by setting the `debug` and `debug-log` flag. +The shim will replace "%ID%" with the container ID, and "%COMMAND%" with the +runsc command (run, boot, etc.) in the path of the `debug-log` flag. + +Find out more about debugging in the [debugging guide](../debugging.md). + +```shell +cat <<EOF | sudo tee /etc/containerd/runsc.toml +[runsc_config] + debug=true + debug-log=/var/log/%ID%/gvisor.%COMMAND%.log +EOF +``` diff --git a/g3doc/user_guide/containerd/containerd_11.md b/g3doc/user_guide/containerd/containerd_11.md new file mode 100644 index 000000000..50befbdf4 --- /dev/null +++ b/g3doc/user_guide/containerd/containerd_11.md @@ -0,0 +1,163 @@ +# Older Versions (containerd 1.1) + +This document describes how to install and run the `gvisor-containerd-shim` +using the untrusted workload CRI extension. This requires `containerd` 1.1 or +later. + +*Note: The untrusted workload CRI extension is deprecated by containerd and +`gvisor-containerd-shim` is maintained on a best-effort basis. If you are using +containerd 1.2+, please see the +[containerd 1.2+ documentation](./quick_start.md) and use +`containerd-shim-runsc-v1`.* + +## Requirements + +- **runsc** and **gvisor-containerd-shim**: See the + [installation guide](/docs/user_guide/install/). +- **containerd**: See the [containerd website](https://containerd.io/) for + information on how to install containerd. + +## Configure containerd + +Create the configuration for the gvisor shim in +`/etc/containerd/gvisor-containerd-shim.toml`: + +```shell +cat <<EOF | sudo tee /etc/containerd/gvisor-containerd-shim.toml +# This is the path to the default runc containerd-shim. +runc_shim = "/usr/local/bin/containerd-shim" +EOF +``` + +Update `/etc/containerd/config.toml`. Be sure to update the path to +`gvisor-containerd-shim` and `runsc` if necessary: + +```shell +cat <<EOF | sudo tee /etc/containerd/config.toml +disabled_plugins = ["restart"] +[plugins.linux] + shim = "/usr/local/bin/gvisor-containerd-shim" + shim_debug = true +[plugins.cri.containerd.untrusted_workload_runtime] + runtime_type = "io.containerd.runtime.v1.linux" + runtime_engine = "/usr/local/bin/runsc" + runtime_root = "/run/containerd/runsc" +EOF +``` + +Restart `containerd`: + +```shell +sudo systemctl restart containerd +``` + +## Usage + +You can run containers in gVisor via containerd's CRI. + +### Install crictl + +Download and install the `crictl` binary: + +```shell +{ +wget https://github.com/kubernetes-sigs/cri-tools/releases/download/v1.13.0/crictl-v1.13.0-linux-amd64.tar.gz +tar xf crictl-v1.13.0-linux-amd64.tar.gz +sudo mv crictl /usr/local/bin +} +``` + +Write the `crictl` configuration file: + +```shell +cat <<EOF | sudo tee /etc/crictl.yaml +runtime-endpoint: unix:///run/containerd/containerd.sock +EOF +``` + +### Create the nginx Sandbox in gVisor + +Pull the nginx image: + +```shell +sudo crictl pull nginx +``` + +Create the sandbox creation request: + +```shell +cat <<EOF | tee sandbox.json +{ + "metadata": { + "name": "nginx-sandbox", + "namespace": "default", + "attempt": 1, + "uid": "hdishd83djaidwnduwk28bcsb" + }, + "annotations": { + "io.kubernetes.cri.untrusted-workload": "true" + }, + "linux": { + }, + "log_directory": "/tmp" +} +EOF +``` + +Create the pod in gVisor: + +```shell +SANDBOX_ID=$(sudo crictl runp sandbox.json) +``` + +### Run the nginx Container in the Sandbox + +Create the nginx container creation request: + +```shell +cat <<EOF | tee container.json +{ + "metadata": { + "name": "nginx" + }, + "image":{ + "image": "nginx" + }, + "log_path":"nginx.0.log", + "linux": { + } +} +EOF +``` + +Create the nginx container: + +```shell +CONTAINER_ID=$(sudo crictl create ${SANDBOX_ID} container.json sandbox.json) +``` + +Start the nginx container: + +```shell +sudo crictl start ${CONTAINER_ID} +``` + +### Validate the container + +Inspect the created pod: + +```shell +sudo crictl inspectp ${SANDBOX_ID} +``` + +Inspect the nginx container: + +```shell +sudo crictl inspect ${CONTAINER_ID} +``` + +Verify that nginx is running in gVisor: + +```shell +sudo crictl exec ${CONTAINER_ID} dmesg | grep -i gvisor +``` diff --git a/g3doc/user_guide/containerd/quick_start.md b/g3doc/user_guide/containerd/quick_start.md new file mode 100644 index 000000000..2f67eecb3 --- /dev/null +++ b/g3doc/user_guide/containerd/quick_start.md @@ -0,0 +1,176 @@ +# Containerd Quick Start + +This document describes how to install and configure `containerd-shim-runsc-v1` +using the containerd runtime handler support on `containerd` 1.2 or later. + +## Requirements + +- **runsc** and **containerd-shim-runsc-v1**: See the + [installation guide](/docs/user_guide/install/). +- **containerd**: See the [containerd website](https://containerd.io/) for + information on how to install containerd. + +## Configure containerd + +Update `/etc/containerd/config.toml`. Make sure `containerd-shim-runsc-v1` is in +`${PATH}` or in the same directory as `containerd` binary. + +```shell +cat <<EOF | sudo tee /etc/containerd/config.toml +disabled_plugins = ["restart"] +[plugins.linux] + shim_debug = true +[plugins.cri.containerd.runtimes.runsc] + runtime_type = "io.containerd.runsc.v1" +EOF +``` + +Restart `containerd`: + +```shell +sudo systemctl restart containerd +``` + +## Usage + +You can run containers in gVisor via containerd's CRI. + +### Install crictl + +Download and install the `crictl`` binary: + +```shell +{ +wget https://github.com/kubernetes-sigs/cri-tools/releases/download/v1.13.0/crictl-v1.13.0-linux-amd64.tar.gz +tar xf crictl-v1.13.0-linux-amd64.tar.gz +sudo mv crictl /usr/local/bin +} +``` + +Write the `crictl` configuration file: + +```shell +cat <<EOF | sudo tee /etc/crictl.yaml +runtime-endpoint: unix:///run/containerd/containerd.sock +EOF +``` + +### Create the nginx sandbox in gVisor + +Pull the nginx image: + +```shell +sudo crictl pull nginx +``` + +Create the sandbox creation request: + +```shell +cat <<EOF | tee sandbox.json +{ + "metadata": { + "name": "nginx-sandbox", + "namespace": "default", + "attempt": 1, + "uid": "hdishd83djaidwnduwk28bcsb" + }, + "linux": { + }, + "log_directory": "/tmp" +} +EOF +``` + +Create the pod in gVisor: + +```shell +SANDBOX_ID=$(sudo crictl runp --runtime runsc sandbox.json) +``` + +### Run the nginx container in the sandbox + +Create the nginx container creation request: + +```shell +cat <<EOF | tee container.json +{ + "metadata": { + "name": "nginx" + }, + "image":{ + "image": "nginx" + }, + "log_path":"nginx.0.log", + "linux": { + } +} +EOF +``` + +Create the nginx container: + +```shell +CONTAINER_ID=$(sudo crictl create ${SANDBOX_ID} container.json sandbox.json) +``` + +Start the nginx container: + +```shell +sudo crictl start ${CONTAINER_ID} +``` + +### Validate the container + +Inspect the created pod: + +```shell +sudo crictl inspectp ${SANDBOX_ID} +``` + +Inspect the nginx container: + +```shell +sudo crictl inspect ${CONTAINER_ID} +``` + +Verify that nginx is running in gVisor: + +```shell +sudo crictl exec ${CONTAINER_ID} dmesg | grep -i gvisor +``` + +### Set up the Kubernetes RuntimeClass + +Install the RuntimeClass for gVisor: + +```shell +cat <<EOF | kubectl apply -f - +apiVersion: node.k8s.io/v1beta1 +kind: RuntimeClass +metadata: + name: gvisor +handler: runsc +EOF +``` + +Create a Pod with the gVisor RuntimeClass: + +```shell +cat <<EOF | kubectl apply -f - +apiVersion: v1 +kind: Pod +metadata: + name: nginx-gvisor +spec: + runtimeClassName: gvisor + containers: + - name: nginx + image: nginx +EOF +``` + +Verify that the Pod is running: + +```shell +kubectl get pod nginx-gvisor -o wide +``` diff --git a/g3doc/user_guide/debugging.md b/g3doc/user_guide/debugging.md index 0525fd5c0..54fdce34f 100644 --- a/g3doc/user_guide/debugging.md +++ b/g3doc/user_guide/debugging.md @@ -129,3 +129,13 @@ go tool pprof -top /usr/local/bin/runsc /tmp/cpu.prof ``` [pprof]: https://github.com/google/pprof/blob/master/doc/README.md + +### Docker Proxy + +When forwarding a port to the container, Docker will likely route traffic +through the [docker-proxy][]. This proxy may make profiling noisy, so it can be +helpful to bypass it. Do so by sending traffic directly to the container IP and +port. e.g., if the `docker0` IP is `192.168.9.1`, the container IP is likely a +subsequent IP, such as `192.168.9.2`. + +[docker-proxy]: https://windsock.io/the-docker-proxy/ diff --git a/g3doc/user_guide/install.md b/g3doc/user_guide/install.md index 9afdd264d..abb9e8582 100644 --- a/g3doc/user_guide/install.md +++ b/g3doc/user_guide/install.md @@ -5,6 +5,68 @@ > Note: gVisor supports only x86\_64 and requires Linux 4.14.77+ > ([older Linux](./networking.md#gso)). +## Install latest release {#install-latest} + +To download and install the latest release manually follow these steps: + +```bash +( + set -e + URL=https://storage.googleapis.com/gvisor/releases/release/latest + wget ${URL}/runsc ${URL}/runsc.sha512 + sha512sum -c runsc.sha512 + rm -f runsc.sha512 + sudo mv runsc /usr/local/bin + sudo chmod a+rx /usr/local/bin/runsc +) +``` + +To install gVisor with Docker, run the following commands: + +```bash +/usr/local/bin/runsc install +sudo systemctl restart docker +docker run --rm --runtime=runsc hello-world +``` + +For more details about using gVisor with Docker, see +[Docker Quick Start](./quick_start/docker.md) + +Note: It is important to copy `runsc` to a location that is readable and +executable to all users, since `runsc` executes itself as user `nobody` to avoid +unnecessary privileges. The `/usr/local/bin` directory is a good place to put +the `runsc` binary. + +## Install from an `apt` repository + +First, appropriate dependencies must be installed to allow `apt` to install +packages via https: + +```bash +sudo apt-get update && \ +sudo apt-get install -y \ + apt-transport-https \ + ca-certificates \ + curl \ + gnupg-agent \ + software-properties-common +``` + +Next, the configure the key used to sign archives and the repository: + +```bash +curl -fsSL https://gvisor.dev/archive.key | sudo apt-key add - +sudo add-apt-repository "deb https://storage.googleapis.com/gvisor/releases release main" +``` + +Now the runsc package can be installed: + +```bash +sudo apt-get update && sudo apt-get install -y runsc +``` + +If you have Docker installed, it will be automatically configured. + ## Versions The `runsc` binaries and repositories are available in multiple versions and @@ -21,12 +83,16 @@ Binaries are available for every commit on the `master` branch, and are available at the following URL: `https://storage.googleapis.com/gvisor/releases/master/latest/runsc` +`https://storage.googleapis.com/gvisor/releases/master/latest/runsc.sha512` -Checksums for the release binary are at: +You can use this link with the steps described in +[Install latest release](#install-latest). -`https://storage.googleapis.com/gvisor/releases/master/latest/runsc.sha512` +For `apt` installation, use the `master` to configure the repository: -For `apt` installation, use the `master` as the `${DIST}` below. +```bash +sudo add-apt-repository "deb https://storage.googleapis.com/gvisor/releases master main" +``` ### Nightly @@ -34,18 +100,22 @@ Nightly releases are built most nights from the master branch, and are available at the following URL: `https://storage.googleapis.com/gvisor/releases/nightly/latest/runsc` - -Checksums for the release binary are at: - `https://storage.googleapis.com/gvisor/releases/nightly/latest/runsc.sha512` +You can use this link with the steps described in +[Install latest release](#install-latest). + Specific nightly releases can be found at: `https://storage.googleapis.com/gvisor/releases/nightly/${yyyy-mm-dd}/runsc` Note that a release may not be available for every day. -For `apt` installation, use the `nightly` as the `${DIST}` below. +For `apt` installation, use the `nightly` to configure the repository: + +```bash +sudo add-apt-repository "deb https://storage.googleapis.com/gvisor/releases nightly main" +``` ### Latest release @@ -53,105 +123,47 @@ The latest official release is available at the following URL: `https://storage.googleapis.com/gvisor/releases/release/latest` -For `apt` installation, use the `release` as the `${DIST}` below. - -### Specific release - -A given release release is available at the following URL: - -`https://storage.googleapis.com/gvisor/releases/release/${yyyymmdd}` - -See the [releases][releases] page for information about specific releases. - -For `apt` installation of a specific release, which may include point updates, -use the date of the release, e.g. `${yyyymmdd}`, as the `${DIST}` below. - -> Note: only newer releases may be available as `apt` repositories. - -### Point release - -A given point release is available at the following URL: - -`https://storage.googleapis.com/gvisor/releases/release/${yyyymmdd}.${rc}` - -Note that `apt` installation of a specific point release is not supported. - -## Install from an `apt` repository +You can use this link with the steps described in +[Install latest release](#install-latest). -First, appropriate dependencies must be installed to allow `apt` to install -packages via https: +For `apt` installation, use the `release` to configure the repository: ```bash -sudo apt-get update && \ -sudo apt-get install -y \ - apt-transport-https \ - ca-certificates \ - curl \ - gnupg-agent \ - software-properties-common +sudo add-apt-repository "deb https://storage.googleapis.com/gvisor/releases release main" ``` -Next, the key used to sign archives should be added to your `apt` keychain: - -```bash -curl -fsSL https://gvisor.dev/archive.key | sudo apt-key add - -``` +### Specific release -Based on the release type, you will need to substitute `${DIST}` below, using -one of: +A given release release is available at the following URL: -* `master`: For HEAD. -* `nightly`: For nightly releases. -* `release`: For the latest release. -* `${yyyymmdd}`: For a specific releases (see above). +`https://storage.googleapis.com/gvisor/releases/release/${yyyymmdd}` -The repository for the release you wish to install should be added: +You can use this link with the steps described in +[Install latest release](#install-latest). -```bash -sudo add-apt-repository "deb https://storage.googleapis.com/gvisor/releases ${DIST} main" -``` +See the [releases](https://github.com/google/gvisor/releases) page for +information about specific releases. -For example, to install the latest official release, you can use: +For `apt` installation of a specific release, which may include point updates, +use the date of the release for repository, e.g. `${yyyymmdd}`. ```bash -sudo add-apt-repository "deb https://storage.googleapis.com/gvisor/releases release main" +sudo add-apt-repository "deb https://storage.googleapis.com/gvisor/releases yyyymmdd main" ``` -Now the runsc package can be installed: - -```bash -sudo apt-get update && sudo apt-get install -y runsc -``` +> Note: only newer releases may be available as `apt` repositories. -If you have Docker installed, it will be automatically configured. +### Point release -## Install directly +A given point release is available at the following URL: -The binary URLs provided above can be used to install directly. For example, the -latest nightly binary can be downloaded, validated, and placed in an appropriate -location by running: +`https://storage.googleapis.com/gvisor/releases/release/${yyyymmdd}.${rc}` -```bash -( - set -e - URL=https://storage.googleapis.com/gvisor/releases/nightly/latest - wget ${URL}/runsc - wget ${URL}/runsc.sha512 - sha512sum -c runsc.sha512 - rm -f runsc.sha512 - sudo mv runsc /usr/local/bin - sudo chown root:root /usr/local/bin/runsc - sudo chmod 0755 /usr/local/bin/runsc -) -``` +You can use this link with the steps described in +[Install latest release](#install-latest). -**It is important to copy this binary to a location that is accessible to all -users, and ensure it is executable by all users**, since `runsc` executes itself -as user `nobody` to avoid unnecessary privileges. The `/usr/local/bin` directory -is a good place to put the `runsc` binary. +Note that `apt` installation of a specific point release is not supported. After installation, try out `runsc` by following the [Docker Quick Start](./quick_start/docker.md) or [OCI Quick Start](./quick_start/oci.md). - -[releases]: https://github.com/google/gvisor/releases diff --git a/g3doc/user_guide/networking.md b/g3doc/user_guide/networking.md index 62def5a90..95f675633 100644 --- a/g3doc/user_guide/networking.md +++ b/g3doc/user_guide/networking.md @@ -2,9 +2,9 @@ [TOC] -gVisor implements its own network stack called netstack. All aspects -of the network stack are handled inside the Sentry — including TCP connection -state, control messages, and packet assembly — keeping it isolated from the host +gVisor implements its own network stack called netstack. All aspects of the +network stack are handled inside the Sentry — including TCP connection state, +control messages, and packet assembly — keeping it isolated from the host network stack. Data link layer packets are written directly to the virtual device inside the network namespace setup by Docker or Kubernetes. diff --git a/g3doc/user_guide/quick_start/docker.md b/g3doc/user_guide/quick_start/docker.md index 6ad594ecc..ee842e453 100644 --- a/g3doc/user_guide/quick_start/docker.md +++ b/g3doc/user_guide/quick_start/docker.md @@ -22,18 +22,6 @@ named "runsc" by default. sudo runsc install ``` -You may also wish to install a runtime entry for debugging. The `runsc install` -command can accept options that will be passed to the runtime when it is invoked -by Docker. - -```bash -sudo runsc install --runtime runsc-debug -- \ - --debug \ - --debug-log=/tmp/runsc-debug.log \ - --strace \ - --log-packets -``` - You must restart the Docker daemon after installing the runtime. Typically this is done via `systemd`: @@ -85,6 +73,21 @@ $ docker run --runtime=runsc -it ubuntu dmesg Note that this is easily replicated by an attacker so applications should never use `dmesg` to verify the runtime in a security sensitive context. +## Options + +You may also wish to install a runtime entry with different options. The `runsc +install` command can accept flags that will be passed to the runtime when it is +invoked by Docker. For example, to install a runtime with debugging enabled, run +the following: + +```bash +sudo runsc install --runtime runsc-debug -- \ + --debug \ + --debug-log=/tmp/runsc-debug.log \ + --strace \ + --log-packets +``` + Next, look at the different options available for gVisor: [platform][platforms], [network][networking], [filesystem][filesystem]. diff --git a/g3doc/user_guide/quick_start/kubernetes.md b/g3doc/user_guide/quick_start/kubernetes.md index f875d8002..395cd4b71 100644 --- a/g3doc/user_guide/quick_start/kubernetes.md +++ b/g3doc/user_guide/quick_start/kubernetes.md @@ -6,17 +6,15 @@ with Kubernetes. ## Using Minikube gVisor can run sandboxed containers in a Kubernetes cluster with Minikube. After -the gVisor addon is enabled, pods with `io.kubernetes.cri.untrusted-workload` +the gVisor addon is enabled, pods with a `gvisor` [Runtime Class][runtimeclass] set to true will execute with `runsc`. Follow [these instructions][minikube] to enable gVisor addon. ## Using Containerd -You can also setup Kubernetes nodes to run pods in gvisor using the -[containerd][containerd] CRI runtime and the `gvisor-containerd-shim`. You can -use either the `io.kubernetes.cri.untrusted-workload` annotation or -[RuntimeClass][runtimeclass] to run Pods with `runsc`. You can find instructions -[here][gvisor-containerd-shim]. +You can also setup Kubernetes nodes to run pods in gVisor using +[containerd][containerd] and the gVisor containerd shim. You can find +instructions in the [Containerd Quick Start][gvisor-containerd]. ## Using GKE Sandbox @@ -31,6 +29,6 @@ WordPress site. You can view the full documentation [here][gke-sandbox-docs]. [gke]: https://cloud.google.com/kubernetes-engine/ [gke-sandbox]: https://cloud.google.com/kubernetes-engine/sandbox/ [gke-sandbox-docs]: https://cloud.google.com/kubernetes-engine/docs/how-to/sandbox-pods -[gvisor-containerd-shim]: https://github.com/google/gvisor-containerd-shim +[gvisor-containerd]: /docs/user_guide/containerd/quick_start/ [runtimeclass]: https://kubernetes.io/docs/concepts/containers/runtime-class/ [wordpress-quick]: /docs/tutorials/kubernetes/ diff --git a/g3doc/user_guide/quick_start/oci.md b/g3doc/user_guide/quick_start/oci.md index 877169145..e7768946b 100644 --- a/g3doc/user_guide/quick_start/oci.md +++ b/g3doc/user_guide/quick_start/oci.md @@ -15,8 +15,8 @@ mkdir bundle cd bundle ``` -Create a root file system for the container. We will use the Docker hello-world -image as the basis for our container. +Create a root file system for the container. We will use the Docker +`hello-world` image as the basis for our container. ```bash mkdir rootfs @@ -24,12 +24,10 @@ docker export $(docker create hello-world) | tar -xf - -C rootfs ``` Next, create an specification file called `config.json` that contains our -container specification. We will update the default command it runs to `/hello` -in the `hello-world` container. +container specification. We tell the container to run the `/hello` program. ```bash -runsc spec -sed -i 's;"sh";"/hello";' config.json +runsc spec -- /hello ``` Finally run the container. diff --git a/g3doc/user_guide/tutorials/BUILD b/g3doc/user_guide/tutorials/BUILD index caae98623..f405349b3 100644 --- a/g3doc/user_guide/tutorials/BUILD +++ b/g3doc/user_guide/tutorials/BUILD @@ -11,16 +11,16 @@ doc( category = "User Guide", permalink = "/docs/tutorials/docker/", subcategory = "Tutorials", - weight = "21", + weight = "10", ) doc( - name = "cni", - src = "cni.md", + name = "docker_compose", + src = "docker-compose.md", category = "User Guide", - permalink = "/docs/tutorials/cni/", + permalink = "/docs/tutorials/docker-compose/", subcategory = "Tutorials", - weight = "22", + weight = "20", ) doc( @@ -33,5 +33,14 @@ doc( ], permalink = "/docs/tutorials/kubernetes/", subcategory = "Tutorials", - weight = "33", + weight = "30", +) + +doc( + name = "cni", + src = "cni.md", + category = "User Guide", + permalink = "/docs/tutorials/cni/", + subcategory = "Tutorials", + weight = "40", ) diff --git a/g3doc/user_guide/tutorials/cni.md b/g3doc/user_guide/tutorials/cni.md index ad6c9fa59..a3507c25b 100644 --- a/g3doc/user_guide/tutorials/cni.md +++ b/g3doc/user_guide/tutorials/cni.md @@ -47,7 +47,7 @@ sudo mkdir -p /etc/cni/net.d sudo sh -c 'cat > /etc/cni/net.d/10-bridge.conf << EOF { - "cniVersion": "0.4.0", + "cniVersion": "0.3.1", "name": "mynet", "type": "bridge", "bridge": "cni0", @@ -65,7 +65,7 @@ EOF' sudo sh -c 'cat > /etc/cni/net.d/99-loopback.conf << EOF { - "cniVersion": "0.4.0", + "cniVersion": "0.3.1", "name": "lo", "type": "loopback" } @@ -128,12 +128,14 @@ sudo mkdir -p rootfs/var/www/html sudo sh -c 'echo "Hello World!" > rootfs/var/www/html/index.html' ``` -Next create the `config.json` specifying the network namespace. `sudo -/usr/local/bin/runsc spec sudo sed -i 's;"sh";"python", "-m", "http.server";' -config.json sudo sed -i "s;\"cwd\": \"/\";\"cwd\": \"/var/www/html\";" -config.json sudo sed -i "s;\"type\": \"network\";\"type\": -\"network\",\n\t\t\t\t\"path\": \"/var/run/netns/${CNI_CONTAINERID}\";" -config.json` +Next create the `config.json` specifying the network namespace. + +``` +sudo /usr/local/bin/runsc spec \ + --cwd /var/www/html \ + --netns /var/run/netns/${CNI_CONTAINERID} \ + -- python -m http.server +``` ## Run the Container diff --git a/g3doc/user_guide/tutorials/docker-compose.md b/g3doc/user_guide/tutorials/docker-compose.md new file mode 100644 index 000000000..3284231f8 --- /dev/null +++ b/g3doc/user_guide/tutorials/docker-compose.md @@ -0,0 +1,100 @@ +# Wordpress with Docker Compose + +This page shows you how to deploy a sample [WordPress][wordpress] site using +[Docker Compose][docker-compose]. + +### Before you begin + +[Follow these instructions][docker-install] to install runsc with Docker. This +document assumes that Docker and Docker Compose are installed and the runtime +name chosen for gVisor is `runsc`. + +### Configuration + +We'll start by creating the `docker-compose.yaml` file to specify our services. +We will specify two services, a `wordpress` service for the Wordpress Apache +server, and a `db` service for MySQL. We will configure Wordpress to connect to +MySQL via the `db` service host name. + +> **Note:** Docker Compose uses it's own network by default and allows services +> to communicate using their service name. Docker Compose does this by setting +> up a DNS server at IP address 127.0.0.11 and configuring containers to use it +> via [resolv.conf][resolv.conf]. This IP is not addressable inside a gVisor +> sandbox so it's important that we set the DNS IP address to the alternative +> `8.8.8.8` and use a network that allows routing to it. See +> [Networking in Compose][compose-networking] for more details. + +> **Note:** The `runtime` field was removed from services in the 3.x version of +> the API in versions of docker-compose < 1.27.0. You will need to write your +> `docker-compose.yaml` file using the 2.x format or use docker-compose >= +> 1.27.0. See this [issue](https://github.com/docker/compose/issues/6239) for +> more details. + +```yaml +version: '2.3' + +services: + db: + image: mysql:5.7 + volumes: + - db_data:/var/lib/mysql + restart: always + environment: + MYSQL_ROOT_PASSWORD: somewordpress + MYSQL_DATABASE: wordpress + MYSQL_USER: wordpress + MYSQL_PASSWORD: wordpress + # All services must be on the same network to communicate. + network_mode: "bridge" + + wordpress: + depends_on: + - db + # When using the "bridge" network specify links. + links: + - db + image: wordpress:latest + ports: + - "8080:80" + restart: always + environment: + WORDPRESS_DB_HOST: db:3306 + WORDPRESS_DB_USER: wordpress + WORDPRESS_DB_PASSWORD: wordpress + WORDPRESS_DB_NAME: wordpress + # Specify the dns address if needed. + dns: + - 8.8.8.8 + # All services must be on the same network to communicate. + network_mode: "bridge" + # Specify the runtime used by Docker. Must be set up in + # /etc/docker/daemon.json. + runtime: "runsc" + +volumes: + db_data: {} +``` + +Once you have a `docker-compose.yaml` in the current directory you can start the +containers: + +```bash +docker-compose up +``` + +Once the containers have started you can access wordpress at +http://localhost:8080. + +Congrats! You now how a working wordpress site up and running using Docker +Compose. + +### What's next + +Learn how to deploy [WordPress with Kubernetes][wordpress-k8s]. + +[docker-compose]: https://docs.docker.com/compose/ +[docker-install]: ../quick_start/docker.md +[wordpress]: https://wordpress.com/ +[resolv.conf]: https://man7.org/linux/man-pages/man5/resolv.conf.5.html +[wordpress-k8s]: kubernetes.md +[compose-networking]: https://docs.docker.com/compose/networking/ diff --git a/g3doc/user_guide/tutorials/docker.md b/g3doc/user_guide/tutorials/docker.md index 705560038..9ca01da2a 100644 --- a/g3doc/user_guide/tutorials/docker.md +++ b/g3doc/user_guide/tutorials/docker.md @@ -60,9 +60,11 @@ Congratulations! You have just deployed a WordPress site using Docker. ### What's next -[Learn how to deploy WordPress with Kubernetes][wordpress-k8s]. +Learn how to deploy WordPress with [Kubernetes][wordpress-k8s] or +[Docker Compose][wordpress-compose]. [docker]: https://www.docker.com/ -[docker-install]: /docs/user_guide/quick_start/docker/ +[docker-install]: ../quick_start/docker.md [wordpress]: https://wordpress.com/ -[wordpress-k8s]: /docs/tutorials/kubernetes/ +[wordpress-k8s]: kubernetes.md +[wordpress-compose]: docker-compose.md diff --git a/g3doc/user_guide/tutorials/kubernetes.md b/g3doc/user_guide/tutorials/kubernetes.md index d2a94b1b7..1ec6e71e9 100644 --- a/g3doc/user_guide/tutorials/kubernetes.md +++ b/g3doc/user_guide/tutorials/kubernetes.md @@ -23,12 +23,12 @@ gcloud beta container node-pools create sandbox-pool --cluster=${CLUSTER_NAME} - If you prefer to use the console, select your cluster and select the **ADD NODE POOL** button: - + Then select the **Image type** with **Containerd** and select **Enable sandbox with gVisor** option. Select other options as you like: - + ### Check that gVisor is enabled @@ -57,47 +57,149 @@ curl -LO https://k8s.io/examples/application/wordpress/mysql-deployment.yaml Add a **spec.template.spec.runtimeClassName** set to **gvisor** to both files, as shown below: -**wordpress-deployment.yaml:** ```yaml apiVersion: v1 kind: Service metadata: -name: wordpress labels: app: wordpress spec: ports: - port: 80 selector: app: -wordpress tier: frontend - -## type: LoadBalancer - -apiVersion: v1 kind: PersistentVolumeClaim metadata: name: wp-pv-claim labels: -app: wordpress spec: accessModes: - ReadWriteOnce resources: requests: - -## storage: 20Gi - -apiVersion: apps/v1 kind: Deployment metadata: name: wordpress labels: app: -wordpress spec: selector: matchLabels: app: wordpress tier: frontend strategy: -type: Recreate template: metadata: labels: app: wordpress tier: frontend spec: -runtimeClassName: gvisor # ADD THIS LINE containers: - image: -wordpress:4.8-apache name: wordpress env: - name: WORDPRESS_DB_HOST value: -wordpress-mysql - name: WORDPRESS_DB_PASSWORD valueFrom: secretKeyRef: name: -mysql-pass key: password ports: - containerPort: 80 name: wordpress -volumeMounts: - name: wordpress-persistent-storage mountPath: /var/www/html -volumes: - name: wordpress-persistent-storage persistentVolumeClaim: claimName: -wp-pv-claim ``` - -**mysql-deployment.yaml:** ```yaml apiVersion: v1 kind: Service metadata: name: -wordpress-mysql labels: app: wordpress spec: ports: - port: 3306 selector: app: -wordpress tier: mysql - -## clusterIP: None - -apiVersion: v1 kind: PersistentVolumeClaim metadata: name: mysql-pv-claim -labels: app: wordpress spec: accessModes: - ReadWriteOnce resources: requests: - -## storage: 20Gi +**wordpress-deployment.yaml:** + +```yaml +apiVersion: v1 +kind: Service +metadata: + name: wordpress + labels: + app: wordpress +spec: + ports: + - port: 80 + selector: + app: wordpress + tier: frontend + type: LoadBalancer +--- +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + name: wp-pv-claim + labels: + app: wordpress +spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 20Gi +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: wordpress + labels: + app: wordpress +spec: + selector: + matchLabels: + app: wordpress + tier: frontend + strategy: + type: Recreate + template: + metadata: + labels: + app: wordpress + tier: frontend + spec: + runtimeClassName: gvisor # ADD THIS LINE + containers: + - image: wordpress:4.8-apache + name: wordpress + env: + - name: WORDPRESS_DB_HOST + value: wordpress-mysql + - name: WORDPRESS_DB_PASSWORD + valueFrom: + secretKeyRef: + name: mysql-pass + key: password + ports: + - containerPort: 80 + name: wordpress + volumeMounts: + - name: wordpress-persistent-storage + mountPath: /var/www/html + volumes: + - name: wordpress-persistent-storage + persistentVolumeClaim: + claimName: wp-pv-claim +``` -apiVersion: apps/v1 kind: Deployment metadata: name: wordpress-mysql labels: -app: wordpress spec: selector: matchLabels: app: wordpress tier: mysql strategy: -type: Recreate template: metadata: labels: app: wordpress tier: mysql spec: -runtimeClassName: gvisor # ADD THIS LINE containers: - image: mysql:5.6 name: -mysql env: - name: MYSQL_ROOT_PASSWORD valueFrom: secretKeyRef: name: mysql-pass -key: password ports: - containerPort: 3306 name: mysql volumeMounts: - name: -mysql-persistent-storage mountPath: /var/lib/mysql volumes: - name: -mysql-persistent-storage persistentVolumeClaim: claimName: mysql-pv-claim ``` +**mysql-deployment.yaml:** + +```yaml +apiVersion: v1 +kind: Service +metadata: + name: wordpress-mysql + labels: + app: wordpress +spec: + ports: + - port: 3306 + selector: + app: wordpress + tier: mysql + clusterIP: None +--- +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + name: mysql-pv-claim + labels: + app: wordpress +spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 20Gi +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: wordpress-mysql + labels: + app: wordpress +spec: + selector: + matchLabels: + app: wordpress + tier: mysql + strategy: + type: Recreate + template: + metadata: + labels: + app: wordpress + tier: mysql + spec: + runtimeClassName: gvisor # ADD THIS LINE + containers: + - image: mysql:5.6 + name: mysql + env: + - name: MYSQL_ROOT_PASSWORD + valueFrom: + secretKeyRef: + name: mysql-pass + key: password + ports: + - containerPort: 3306 + name: mysql + volumeMounts: + - name: mysql-persistent-storage + mountPath: /var/lib/mysql + volumes: + - name: mysql-persistent-storage + persistentVolumeClaim: + claimName: mysql-pv-claim +``` Note that apart from `runtimeClassName: gvisor`, nothing else about the Deployment has is changed. |