summaryrefslogtreecommitdiffhomepage
path: root/shim
diff options
context:
space:
mode:
authorgVisor bot <gvisor-bot@google.com>2020-07-13 16:10:58 -0700
committergVisor bot <gvisor-bot@google.com>2020-07-13 16:10:58 -0700
commitc81ac8ec3b78f898e75830a93b5ed3085b785fd6 (patch)
tree03042d0bb4b544533d08da6625069d3e6c10d657 /shim
parent59a5479409094b141a60cfcc65f0a53d7871e2e1 (diff)
parent2afff44403e046078301de39f0252bb57fc018c7 (diff)
Merge pull request #2672 from amscanne:shim-integrated
PiperOrigin-RevId: 321053634
Diffstat (limited to 'shim')
-rw-r--r--shim/BUILD15
-rw-r--r--shim/README.md11
-rw-r--r--shim/runsc.toml6
-rw-r--r--shim/v1/BUILD41
-rw-r--r--shim/v1/README.md50
-rw-r--r--shim/v1/api.go24
-rw-r--r--shim/v1/config.go40
-rw-r--r--shim/v1/main.go265
-rw-r--r--shim/v2/BUILD29
-rw-r--r--shim/v2/README.md91
-rw-r--r--shim/v2/main.go26
-rw-r--r--shim/v2/runtime-handler-shim-v2-quickstart.md251
12 files changed, 849 insertions, 0 deletions
diff --git a/shim/BUILD b/shim/BUILD
new file mode 100644
index 000000000..e581618b2
--- /dev/null
+++ b/shim/BUILD
@@ -0,0 +1,15 @@
+load("//tools:defs.bzl", "pkg_tar")
+
+package(licenses = ["notice"])
+
+pkg_tar(
+ name = "config",
+ srcs = [
+ "runsc.toml",
+ ],
+ mode = "0644",
+ package_dir = "/etc/containerd",
+ visibility = [
+ "//runsc:__pkg__",
+ ],
+)
diff --git a/shim/README.md b/shim/README.md
new file mode 100644
index 000000000..c6824ebdc
--- /dev/null
+++ b/shim/README.md
@@ -0,0 +1,11 @@
+# gVisor Containerd shims
+
+There are various shims supported for differt versions of
+[containerd][containerd].
+
+- [Configure gvisor-containerd-shim (shim v1) (containerd &le; 1.2)](v1/configure-gvisor-containerd-shim.md)
+- [Runtime Handler/RuntimeClass Quick Start (containerd >= 1.2)](v2/runtime-handler-quickstart.md)
+- [Runtime Handler/RuntimeClass Quick Start (shim v2) (containerd >= 1.2)](v2/runtime-handler-shim-v2-quickstart.md)
+- [Configure containerd-shim-runsc-v1 (shim v2) (containerd >= 1.3)](v2/configure-containerd-shim-runsc-v1.md)
+
+[containerd]: https://github.com/containerd/containerd
diff --git a/shim/runsc.toml b/shim/runsc.toml
new file mode 100644
index 000000000..e1c7de1bb
--- /dev/null
+++ b/shim/runsc.toml
@@ -0,0 +1,6 @@
+# This is an example configuration file for runsc.
+#
+# By default, it will be parsed from /etc/containerd/runsc.toml, but see the
+# static path configured in v1/main.go. Note that the configuration mechanism
+# for newer container shim versions is different: see the documentation in v2.
+[runsc_config]
diff --git a/shim/v1/BUILD b/shim/v1/BUILD
new file mode 100644
index 000000000..7b837630c
--- /dev/null
+++ b/shim/v1/BUILD
@@ -0,0 +1,41 @@
+load("//tools:defs.bzl", "go_binary")
+load("//website:defs.bzl", "doc")
+
+package(licenses = ["notice"])
+
+go_binary(
+ name = "gvisor-containerd-shim",
+ srcs = [
+ "api.go",
+ "config.go",
+ "main.go",
+ ],
+ static = True,
+ visibility = [
+ "//visibility:public",
+ ],
+ deps = [
+ "//pkg/shim/runsc",
+ "//pkg/shim/v1/shim",
+ "@com_github_burntsushi_toml//:go_default_library",
+ "@com_github_containerd_containerd//events:go_default_library",
+ "@com_github_containerd_containerd//namespaces:go_default_library",
+ "@com_github_containerd_containerd//runtime/v1/shim/v1:go_default_library",
+ "@com_github_containerd_containerd//sys:go_default_library",
+ "@com_github_containerd_containerd//sys/reaper:go_default_library",
+ "@com_github_containerd_ttrpc//:go_default_library",
+ "@com_github_containerd_typeurl//:go_default_library",
+ "@com_github_gogo_protobuf//types:go_default_library",
+ "@org_golang_x_sys//unix:go_default_library",
+ ],
+)
+
+doc(
+ name = "doc",
+ src = "README.md",
+ category = "User Guide",
+ permalink = "/docs/user_guide/gvisor-containerd-shim/",
+ subcategory = "Advanced",
+ visibility = ["//website:__pkg__"],
+ weight = "93",
+)
diff --git a/shim/v1/README.md b/shim/v1/README.md
new file mode 100644
index 000000000..7aa4513a1
--- /dev/null
+++ b/shim/v1/README.md
@@ -0,0 +1,50 @@
+# gvisor-containerd-shim
+
+> Note: This shim version is supported only for containerd versions less than
+> 1.2. If you are using a containerd version greater than or equal to 1.2, then
+> please use `containerd-shim-runsc-v1` (Shim API v1).
+>
+> This containerd shim is supported only in a best-effort capacity.
+
+This document describes how to configure and use `gvisor-containerd-shim`.
+
+## Containerd Configuration
+
+To use this shim, you must configure `/etc/containerd/config.toml` as follows:
+
+```
+[plugins.linux]
+ shim = "/usr/bin/gvisor-containerd-shim"
+[plugins.cri.containerd.runtimes.gvisor]
+ runtime_type = "io.containerd.runtime.v1.linux"
+ runtime_engine = "/usr/bin/runsc"
+ runtime_root = "/run/containerd/runsc"
+```
+
+In order to pick-up the new configuration, you may need to restart containerd:
+
+```shell
+sudo systemctl restart containerd
+```
+
+## Shim Confguration
+
+The shim configuration is stored in `/etc/containerd/runsc.toml`. The
+configuration file supports two values.
+
+* `runc_shim`: The path to the runc shim. This is used by
+ `gvisor-containerd-shim` to run standard containers.
+
+* `runsc_config`: This is a set of key/value pairs that are converted into
+ `runsc` command line flags. You can learn more about which flags are
+ available by running `runsc flags`.
+
+For example, a configuration might look as follows:
+
+```
+runc_shim = "/usr/local/bin/containerd-shim"
+[runsc_config]
+platform = "kvm"
+debug = true
+debug-log = /var/log/%ID%/gvisor/
+```
diff --git a/shim/v1/api.go b/shim/v1/api.go
new file mode 100644
index 000000000..2444d23f1
--- /dev/null
+++ b/shim/v1/api.go
@@ -0,0 +1,24 @@
+// Copyright 2018 The containerd Authors.
+// Copyright 2019 The gVisor Authors.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// https://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package main
+
+import (
+ shim "github.com/containerd/containerd/runtime/v1/shim/v1"
+)
+
+type KillRequest = shim.KillRequest
+
+var registerShimService = shim.RegisterShimService
diff --git a/shim/v1/config.go b/shim/v1/config.go
new file mode 100644
index 000000000..a72cc7754
--- /dev/null
+++ b/shim/v1/config.go
@@ -0,0 +1,40 @@
+// Copyright 2018 The gVisor Authors.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// https://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package main
+
+import "github.com/BurntSushi/toml"
+
+// config is the configuration for gvisor containerd shim.
+type config struct {
+ // RuncShim is the shim binary path for standard containerd-shim for runc.
+ // When the runtime is `runc`, gvisor containerd shim will exec current
+ // process to standard containerd-shim. This is a work around for containerd
+ // 1.1. In containerd 1.2, containerd will choose different containerd-shims
+ // based on runtime.
+ RuncShim string `toml:"runc_shim"`
+ // RunscConfig is configuration for runsc. The key value will be converted
+ // to runsc flags --key=value directly.
+ RunscConfig map[string]string `toml:"runsc_config"`
+}
+
+// loadConfig load gvisor containerd shim config from config file.
+func loadConfig(path string) (*config, error) {
+ var c config
+ _, err := toml.DecodeFile(path, &c)
+ if err != nil {
+ return &c, err
+ }
+ return &c, nil
+}
diff --git a/shim/v1/main.go b/shim/v1/main.go
new file mode 100644
index 000000000..3159923af
--- /dev/null
+++ b/shim/v1/main.go
@@ -0,0 +1,265 @@
+// Copyright 2018 The containerd Authors.
+// Copyright 2019 The gVisor Authors.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// https://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package main
+
+import (
+ "bytes"
+ "context"
+ "flag"
+ "fmt"
+ "log"
+ "net"
+ "os"
+ "os/exec"
+ "os/signal"
+ "path/filepath"
+ "strings"
+ "sync"
+ "syscall"
+
+ "github.com/containerd/containerd/events"
+ "github.com/containerd/containerd/namespaces"
+ "github.com/containerd/containerd/sys"
+ "github.com/containerd/containerd/sys/reaper"
+ "github.com/containerd/ttrpc"
+ "github.com/containerd/typeurl"
+ "github.com/gogo/protobuf/types"
+ "golang.org/x/sys/unix"
+
+ "gvisor.dev/gvisor/pkg/shim/runsc"
+ "gvisor.dev/gvisor/pkg/shim/v1/shim"
+)
+
+var (
+ debugFlag bool
+ namespaceFlag string
+ socketFlag string
+ addressFlag string
+ workdirFlag string
+ runtimeRootFlag string
+ containerdBinaryFlag string
+ shimConfigFlag string
+)
+
+// Containerd defaults to runc, unless another runtime is explicitly specified.
+// We keep the same default to make the default behavior consistent.
+const defaultRoot = "/run/containerd/runc"
+
+func init() {
+ flag.BoolVar(&debugFlag, "debug", false, "enable debug output in logs")
+ flag.StringVar(&namespaceFlag, "namespace", "", "namespace that owns the shim")
+ flag.StringVar(&socketFlag, "socket", "", "abstract socket path to serve")
+ flag.StringVar(&addressFlag, "address", "", "grpc address back to main containerd")
+ flag.StringVar(&workdirFlag, "workdir", "", "path used to storge large temporary data")
+ flag.StringVar(&runtimeRootFlag, "runtime-root", defaultRoot, "root directory for the runtime")
+
+ // Currently, the `containerd publish` utility is embedded in the
+ // daemon binary. The daemon invokes `containerd-shim
+ // -containerd-binary ...` with its own os.Executable() path.
+ flag.StringVar(&containerdBinaryFlag, "containerd-binary", "containerd", "path to containerd binary (used for `containerd publish`)")
+ flag.StringVar(&shimConfigFlag, "config", "/etc/containerd/runsc.toml", "path to the shim configuration file")
+}
+
+func main() {
+ flag.Parse()
+
+ // This is a hack. Exec current process to run standard containerd-shim
+ // if runtime root is not `runsc`. We don't need this for shim v2 api.
+ if filepath.Base(runtimeRootFlag) != "runsc" {
+ if err := executeRuncShim(); err != nil {
+ fmt.Fprintf(os.Stderr, "gvisor-containerd-shim: %s\n", err)
+ os.Exit(1)
+ }
+ }
+
+ // Run regular shim if needed.
+ if err := executeShim(); err != nil {
+ fmt.Fprintf(os.Stderr, "gvisor-containerd-shim: %s\n", err)
+ os.Exit(1)
+ }
+}
+
+// executeRuncShim execs current process to a containerd-shim process and
+// retains all flags and envs.
+func executeRuncShim() error {
+ c, err := loadConfig(shimConfigFlag)
+ if err != nil && !os.IsNotExist(err) {
+ return fmt.Errorf("failed to load shim config: %w", err)
+ }
+ shimPath := c.RuncShim
+ if shimPath == "" {
+ shimPath, err = exec.LookPath("containerd-shim")
+ if err != nil {
+ return fmt.Errorf("lookup containerd-shim failed: %w", err)
+ }
+ }
+
+ args := append([]string{shimPath}, os.Args[1:]...)
+ if err := syscall.Exec(shimPath, args, os.Environ()); err != nil {
+ return fmt.Errorf("exec containerd-shim @ %q failed: %w", shimPath, err)
+ }
+ return nil
+}
+
+func executeShim() error {
+ // start handling signals as soon as possible so that things are
+ // properly reaped or if runtime exits before we hit the handler.
+ signals, err := setupSignals()
+ if err != nil {
+ return err
+ }
+ path, err := os.Getwd()
+ if err != nil {
+ return err
+ }
+ server, err := ttrpc.NewServer(ttrpc.WithServerHandshaker(ttrpc.UnixSocketRequireSameUser()))
+ if err != nil {
+ return fmt.Errorf("failed creating server: %w", err)
+ }
+ c, err := loadConfig(shimConfigFlag)
+ if err != nil && !os.IsNotExist(err) {
+ return fmt.Errorf("failed to load shim config: %w", err)
+ }
+ sv, err := shim.NewService(
+ shim.Config{
+ Path: path,
+ Namespace: namespaceFlag,
+ WorkDir: workdirFlag,
+ RuntimeRoot: runtimeRootFlag,
+ RunscConfig: c.RunscConfig,
+ },
+ &remoteEventsPublisher{address: addressFlag},
+ )
+ if err != nil {
+ return err
+ }
+ registerShimService(server, sv)
+ if err := serve(server, socketFlag); err != nil {
+ return err
+ }
+ return handleSignals(signals, server, sv)
+}
+
+// serve serves the ttrpc API over a unix socket at the provided path this
+// function does not block.
+func serve(server *ttrpc.Server, path string) error {
+ var (
+ l net.Listener
+ err error
+ )
+ if path == "" {
+ l, err = net.FileListener(os.NewFile(3, "socket"))
+ path = "[inherited from parent]"
+ } else {
+ if len(path) > 106 {
+ return fmt.Errorf("%q: unix socket path too long (> 106)", path)
+ }
+ l, err = net.Listen("unix", "\x00"+path)
+ }
+ if err != nil {
+ return err
+ }
+ go func() {
+ defer l.Close()
+ err := server.Serve(context.Background(), l)
+ if err != nil && !strings.Contains(err.Error(), "use of closed network connection") {
+ log.Fatalf("ttrpc server failure: %v", err)
+ }
+ }()
+ return nil
+}
+
+// setupSignals creates a new signal handler for all signals and sets the shim
+// as a sub-reaper so that the container processes are reparented.
+func setupSignals() (chan os.Signal, error) {
+ signals := make(chan os.Signal, 32)
+ signal.Notify(signals, unix.SIGTERM, unix.SIGINT, unix.SIGCHLD, unix.SIGPIPE)
+ // make sure runc is setup to use the monitor for waiting on processes.
+ // TODO(random-liu): Move shim/reaper.go to a separate package.
+ runsc.Monitor = reaper.Default
+ // Set the shim as the subreaper for all orphaned processes created by
+ // the container.
+ if err := unix.Prctl(unix.PR_SET_CHILD_SUBREAPER, 1, 0, 0, 0); err != nil {
+ return nil, err
+ }
+ return signals, nil
+}
+
+func handleSignals(signals chan os.Signal, server *ttrpc.Server, sv *shim.Service) error {
+ var (
+ termOnce sync.Once
+ done = make(chan struct{})
+ )
+
+ for {
+ select {
+ case <-done:
+ return nil
+ case s := <-signals:
+ switch s {
+ case unix.SIGCHLD:
+ if _, err := sys.Reap(false); err != nil {
+ log.Printf("reap error: %v", err)
+ }
+ case unix.SIGTERM, unix.SIGINT:
+ go termOnce.Do(func() {
+ ctx := context.TODO()
+ if err := server.Shutdown(ctx); err != nil {
+ log.Printf("failed to shutdown server: %v", err)
+ }
+ // Ensure our child is dead if any.
+ sv.Kill(ctx, &KillRequest{
+ Signal: uint32(syscall.SIGKILL),
+ All: true,
+ })
+ sv.Delete(context.Background(), &types.Empty{})
+ close(done)
+ })
+ case unix.SIGPIPE:
+ }
+ }
+ }
+}
+
+type remoteEventsPublisher struct {
+ address string
+}
+
+func (l *remoteEventsPublisher) Publish(ctx context.Context, topic string, event events.Event) error {
+ ns, _ := namespaces.Namespace(ctx)
+ encoded, err := typeurl.MarshalAny(event)
+ if err != nil {
+ return err
+ }
+ data, err := encoded.Marshal()
+ if err != nil {
+ return err
+ }
+ cmd := exec.CommandContext(ctx, containerdBinaryFlag, "--address", l.address, "publish", "--topic", topic, "--namespace", ns)
+ cmd.Stdin = bytes.NewReader(data)
+ c, err := reaper.Default.Start(cmd)
+ if err != nil {
+ return err
+ }
+ status, err := reaper.Default.Wait(cmd, c)
+ if err != nil {
+ return fmt.Errorf("failed to publish event: %w", err)
+ }
+ if status != 0 {
+ return fmt.Errorf("failed to publish event: status %d", status)
+ }
+ return nil
+}
diff --git a/shim/v2/BUILD b/shim/v2/BUILD
new file mode 100644
index 000000000..ae4705935
--- /dev/null
+++ b/shim/v2/BUILD
@@ -0,0 +1,29 @@
+load("//tools:defs.bzl", "go_binary")
+load("//website:defs.bzl", "doc")
+
+package(licenses = ["notice"])
+
+go_binary(
+ name = "containerd-shim-runsc-v1",
+ srcs = [
+ "main.go",
+ ],
+ static = True,
+ visibility = [
+ "//visibility:public",
+ ],
+ deps = [
+ "//pkg/shim/v2",
+ "@com_github_containerd_containerd//runtime/v2/shim:go_default_library",
+ ],
+)
+
+doc(
+ name = "doc",
+ src = "README.md",
+ category = "User Guide",
+ permalink = "/docs/user_guide/containerd-shim-runsc-v1/",
+ subcategory = "Advanced",
+ visibility = ["//website:__pkg__"],
+ weight = "92",
+)
diff --git a/shim/v2/README.md b/shim/v2/README.md
new file mode 100644
index 000000000..2aa7c21e3
--- /dev/null
+++ b/shim/v2/README.md
@@ -0,0 +1,91 @@
+# containerd-shim-runsc-v1
+
+> Note: This shim version is the recommended shim for containerd versions
+> greater than or equal to 1.2. For older versions of containerd, use
+> `gvisor-containerd-shim`.
+
+This document describes how to configure and use `containerd-shim-runsc-v1`.
+
+## Configuring Containerd 1.2
+
+To configure containerd 1.2 to use this shim, add the runtime to
+`/etc/containerd/config.toml` as follows:
+
+```
+[plugins.cri.containerd.runtimes.runsc]
+ runtime_type = "io.containerd.runsc.v1"
+ runtime_root = "/run/containerd/runsc"
+[plugins.cri.containerd.runtimes.runsc.options]
+ TypeUrl = "io.containerd.runsc.v1.options"
+```
+
+The configuration will optionally loaded from a file named `config.toml` in the
+`runtime_root` configured above.
+
+In order to pick up the new configuration, you may need to restart containerd:
+
+```shell
+sudo systemctl restart containerd
+```
+
+## Configuring Containerd 1.3 and above
+
+To configure containerd 1.3 to use this shim, add the runtime to
+`/etc/containerd/config.toml` as follows:
+
+```
+[plugins.cri.containerd.runtimes.runsc]
+ runtime_type = "io.containerd.runsc.v1"
+[plugins.cri.containerd.runtimes.runsc.options]
+ TypeUrl = "io.containerd.runsc.v1.options"
+ ConfigPath = "/etc/containerd/runsc.toml"
+```
+
+The `ConfigPath` above will be used to provide a pointer to the configuration
+file to be loaded.
+
+> Note that there will be configuration file loaded if `ConfigPath` is not set.
+
+In order to pick up the new configuration, you may need to restart containerd:
+
+```shell
+sudo systemctl restart containerd
+```
+
+## Shim Confguration
+
+The shim configuration may carry the following options:
+
+* `shim_cgroup`: The cgroup to use for the shim itself.
+* `io_uid`: The UID to use for pipes.
+* `ui_gid`: The GID to use for pipes.
+* `binary_name`: The runtime binary name (defaults to `runsc`).
+* `root`: The root directory for the runtime.
+* `runsc_config`: A dictionary of key-value pairs that will be passed to the
+ runtime as arguments.
+
+### Example: Enable the KVM platform
+
+gVisor enables the use of a number of platforms. This example shows how to
+configure `containerd-shim-runsc-v1` to use gVisor with the KVM platform:
+
+```shell
+cat <<EOF | sudo tee /etc/containerd/runsc.toml
+[runsc_config]
+platform = "kvm"
+EOF
+```
+
+### Example: Enable gVisor debug logging
+
+gVisor debug logging can be enabled by setting the `debug` and `debug-log` flag.
+The shim will replace "%ID%" with the container ID in the path of the
+`debug-log` flag.
+
+```shell
+cat <<EOF | sudo tee /etc/containerd/runsc.toml
+[runsc_config]
+debug = true
+debug-log = /var/log/%ID%/gvisor.log
+EOF
+```
diff --git a/shim/v2/main.go b/shim/v2/main.go
new file mode 100644
index 000000000..753871eea
--- /dev/null
+++ b/shim/v2/main.go
@@ -0,0 +1,26 @@
+// Copyright 2018 The containerd Authors.
+// Copyright 2019 The gVisor Authors.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// https://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package main
+
+import (
+ "github.com/containerd/containerd/runtime/v2/shim"
+
+ "gvisor.dev/gvisor/pkg/shim/v2"
+)
+
+func main() {
+ shim.Run("io.containerd.runsc.v1", v2.New)
+}
diff --git a/shim/v2/runtime-handler-shim-v2-quickstart.md b/shim/v2/runtime-handler-shim-v2-quickstart.md
new file mode 100644
index 000000000..3b88ca74b
--- /dev/null
+++ b/shim/v2/runtime-handler-shim-v2-quickstart.md
@@ -0,0 +1,251 @@
+# Runtime Handler Quickstart (Shim V2)
+
+This document describes how to install and run `containerd-shim-runsc-v1` using
+the containerd runtime handler support. This requires containerd 1.2 or later.
+
+## Requirements
+
+- **runsc**: See the [gVisor documentation](https://github.com/google/gvisor)
+ for information on how to install runsc.
+- **containerd**: See the [containerd website](https://containerd.io/) for
+ information on how to install containerd.
+
+## Install
+
+### Install containerd-shim-runsc-v1
+
+1. Build and install `containerd-shim-runsc-v1`.
+
+<!-- TODO: Use a release once we have one available. -->
+
+[embedmd]:# (../test/e2e/shim-install.sh shell /{ # Step 1\(dev\)/ /^}/)
+
+```shell
+{ # Step 1(dev): Build and install gvisor-containerd-shim and containerd-shim-runsc-v1
+ make
+ sudo make install
+}
+```
+
+### Configure containerd
+
+1. Update `/etc/containerd/config.toml`. Make sure `containerd-shim-runsc-v1`
+ is in `${PATH}`.
+
+[embedmd]:# (../test/e2e/runtime-handler-shim-v2/install.sh shell /{ # Step 1/ /^}/)
+
+```shell
+{ # Step 1: Create containerd config.toml
+cat <<EOF | sudo tee /etc/containerd/config.toml
+disabled_plugins = ["restart"]
+[plugins.linux]
+ shim_debug = true
+[plugins.cri.containerd.runtimes.runsc]
+ runtime_type = "io.containerd.runsc.v1"
+EOF
+}
+```
+
+1. Restart `containerd`
+
+```shell
+sudo systemctl restart containerd
+```
+
+## Usage
+
+You can run containers in gVisor via containerd's CRI.
+
+### Install crictl
+
+1. Download and install the crictl binary:
+
+[embedmd]:# (../test/e2e/crictl-install.sh shell /{ # Step 1/ /^}/)
+
+```shell
+{ # Step 1: Download crictl
+wget https://github.com/kubernetes-sigs/cri-tools/releases/download/v1.13.0/crictl-v1.13.0-linux-amd64.tar.gz
+tar xf crictl-v1.13.0-linux-amd64.tar.gz
+sudo mv crictl /usr/local/bin
+}
+```
+
+1. Write the crictl configuration file
+
+[embedmd]:# (../test/e2e/crictl-install.sh shell /{ # Step 2/ /^}/)
+
+```shell
+{ # Step 2: Configure crictl
+cat <<EOF | sudo tee /etc/crictl.yaml
+runtime-endpoint: unix:///run/containerd/containerd.sock
+EOF
+}
+```
+
+### Create the nginx Sandbox in gVisor
+
+1. Pull the nginx image
+
+[embedmd]:# (../test/e2e/runtime-handler/usage.sh shell /{ # Step 1/ /^}/)
+
+```shell
+{ # Step 1: Pull the nginx image
+sudo crictl pull nginx
+}
+```
+
+1. Create the sandbox creation request
+
+[embedmd]:# (../test/e2e/runtime-handler/usage.sh shell /{ # Step 2/ /^EOF\n}/)
+
+```shell
+{ # Step 2: Create sandbox.json
+cat <<EOF | tee sandbox.json
+{
+ "metadata": {
+ "name": "nginx-sandbox",
+ "namespace": "default",
+ "attempt": 1,
+ "uid": "hdishd83djaidwnduwk28bcsb"
+ },
+ "linux": {
+ },
+ "log_directory": "/tmp"
+}
+EOF
+}
+```
+
+1. Create the pod in gVisor
+
+[embedmd]:# (../test/e2e/runtime-handler/usage.sh shell /{ # Step 3/ /^}/)
+
+```shell
+{ # Step 3: Create the sandbox
+SANDBOX_ID=$(sudo crictl runp --runtime runsc sandbox.json)
+}
+```
+
+### Run the nginx Container in the Sandbox
+
+1. Create the nginx container creation request
+
+[embedmd]:# (../test/e2e/run-container.sh shell /{ # Step 1/ /^EOF\n}/)
+
+```shell
+{ # Step 1: Create nginx container config
+cat <<EOF | tee container.json
+{
+ "metadata": {
+ "name": "nginx"
+ },
+ "image":{
+ "image": "nginx"
+ },
+ "log_path":"nginx.0.log",
+ "linux": {
+ }
+}
+EOF
+}
+```
+
+1. Create the nginx container
+
+[embedmd]:# (../test/e2e/run-container.sh shell /{ # Step 2/ /^}/)
+
+```shell
+{ # Step 2: Create nginx container
+CONTAINER_ID=$(sudo crictl create ${SANDBOX_ID} container.json sandbox.json)
+}
+```
+
+1. Start the nginx container
+
+[embedmd]:# (../test/e2e/run-container.sh shell /{ # Step 3/ /^}/)
+
+```shell
+{ # Step 3: Start nginx container
+sudo crictl start ${CONTAINER_ID}
+}
+```
+
+### Validate the container
+
+1. Inspect the created pod
+
+[embedmd]:# (../test/e2e/validate.sh shell /{ # Step 1/ /^}/)
+
+```shell
+{ # Step 1: Inspect the pod
+sudo crictl inspectp ${SANDBOX_ID}
+}
+```
+
+1. Inspect the nginx container
+
+[embedmd]:# (../test/e2e/validate.sh shell /{ # Step 2/ /^}/)
+
+```shell
+{ # Step 2: Inspect the container
+sudo crictl inspect ${CONTAINER_ID}
+}
+```
+
+1. Verify that nginx is running in gVisor
+
+[embedmd]:# (../test/e2e/validate.sh shell /{ # Step 3/ /^}/)
+
+```shell
+{ # Step 3: Check dmesg
+sudo crictl exec ${CONTAINER_ID} dmesg | grep -i gvisor
+}
+```
+
+### Set up the Kubernetes Runtime Class
+
+1. Install the Runtime Class for gVisor
+
+[embedmd]:# (../test/e2e/runtimeclass-install.sh shell /{ # Step 1/ /^}/)
+
+```shell
+{ # Step 1: Install a RuntimeClass
+cat <<EOF | kubectl apply -f -
+apiVersion: node.k8s.io/v1beta1
+kind: RuntimeClass
+metadata:
+ name: gvisor
+handler: runsc
+EOF
+}
+```
+
+1. Create a Pod with the gVisor Runtime Class
+
+[embedmd]:# (../test/e2e/runtimeclass-install.sh shell /{ # Step 2/ /^}/)
+
+```shell
+{ # Step 2: Create a pod
+cat <<EOF | kubectl apply -f -
+apiVersion: v1
+kind: Pod
+metadata:
+ name: nginx-gvisor
+spec:
+ runtimeClassName: gvisor
+ containers:
+ - name: nginx
+ image: nginx
+EOF
+}
+```
+
+1. Verify that the Pod is running
+
+[embedmd]:# (../test/e2e/runtimeclass-install.sh shell /{ # Step 3/ /^}/)
+
+```shell
+{ # Step 3: Get the pod
+kubectl get pod nginx-gvisor -o wide
+}
+```