1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
|
+++
title = "Debugging"
weight = 120
+++
To enable debug and system call logging, add the `runtimeArgs` below to your
[Docker](../docker/) configuration (`/etc/docker/daemon.json`):
```json
{
"runtimes": {
"runsc": {
"path": "/usr/local/bin/runsc",
"runtimeArgs": [
"--debug-log=/tmp/runsc/",
"--debug",
"--strace"
]
}
}
}
```
You may also want to pass `--log-packets` to troubleshoot network problems. Then
restart the Docker daemon:
```bash
sudo systemctl restart docker
```
Run your container again, and inspect the files under `/tmp/runsc`. The log file
with name `boot` will contain the strace logs from your application, which can
be useful for identifying missing or broken system calls in gVisor.
## Stack traces
The command `runsc debug --stacks` collects stack traces while the sandbox is
running which can be useful to troubleshoot issues or just to learn more about
gVisor. It connects to the sandbox process, collects a stack dump, and writes
it to the console. For example:
```bash
docker run --runtime=runsc --rm -d alpine sh -c "while true; do echo running; sleep .1; done"
63254c6ab3a6989623fa1fb53616951eed31ac605a2637bb9ddba5d8d404b35b
sudo runsc --root /var/run/docker/runtime-runsc/moby debug --stacks 63254c6ab3a6989623fa1fb53616951eed31ac605a2637bb9ddba5d8d404b35b
```
> Note: `--root` variable is provided by docker and is normally set to
> `/var/run/docker/runtime-[runtime-name]/moby`. If in doubt, `--root` is logged to
> `runsc` logs.
## Profiling
`runsc` integrates with Go profiling tools and gives you easy commands to profile
CPU and heap usage. First you need to enable `--profile` in the command line options
before starting the container:
```json
{
"runtimes": {
"runsc-prof": {
"path": "/usr/local/bin/runsc",
"runtimeArgs": [
"--profile"
]
}
}
}
```
> Note: Enabling profiling loosens the seccomp protection added to the sandbox,
> and should not be run in production under normal circumstances.
Then restart docker to refresh the runtime options. While the container is running,
execute `runsc debug` to collect profile information and save to a file. Here are
the options available:
* **--profile-heap:** Generates heap profile to the speficied file.
* **--profile-cpu:** Enables CPU profiler, waits for `--profile-delay` seconds
and generates CPU profile to the speficied file.
For example:
```bash
docker run --runtime=runsc-prof --rm -d alpine sleep 1000
63254c6ab3a6989623fa1fb53616951eed31ac605a2637bb9ddba5d8d404b35b
sudo runsc --root /var/run/docker/runtime-runsc-prof/moby debug --profile-heap=/tmp/heap.prof 63254c6ab3a6989623fa1fb53616951eed31ac605a2637bb9ddba5d8d404b35b
sudo runsc --root /var/run/docker/runtime-runsc-prof/moby debug --profile-cpu=/tmp/cpu.prof --profile-delay=30 63254c6ab3a6989623fa1fb53616951eed31ac605a2637bb9ddba5d8d404b35b
```
The resulting files can be opened using `go tool pprof` or [pprof]. The examples
below create image file (`.svg`) with the heap profile and writes the top
functions using CPU to the console:
```bash
go tool pprof -svg /usr/local/bin/runsc /tmp/heap.prof
go tool pprof -top /usr/local/bin/runsc /tmp/cpu.prof
```
[pprof]: https://github.com/google/pprof/blob/master/doc/README.md
|
