| Age | Commit message (Collapse) | Author |
|---|
|
|
|
Runsc build benchmark's mutex profile shows that we are wasting roughly 25-30
seconds waiting for filesystem.renameMu to get unlocked. Earlier
checkCachingLocked required the renameMu to be locked for writing. This is a
filesystem wide lock which puts all other filesystem operations on hold and
hence is really expensive. Something to note is that all path resolution
operations hold renameMu for reading.
With this change, we allow to check for caching without even holding renameMu.
This change introduces more fine grained locks (fs.cacheMu and dentry.cachingMu)
which protect the cache (removing the requirement to hold renameMu for writing
to modify the cache) and synchronize concurrent dentry caching attempts on a per
dentry basis. We still require to hold renameMu for writing while destroying
dentries and evicting from the cache but this still significantly reduces the
write locking critical section.
Local benchmarking showed that this improved runsc build benchmark time by 4-5%.
Across 6 runs, without this change it took 310.9025 seconds to build runsc
while with this change it took 296.127 seconds.
Runsc build benchmark's mutex profile: https://gvisor.dev/profile/gvisor-buildkite/78a3f968-36ca-4944-93f7-77a8792d56b4/28a1d260-790b-4a9e-94da-a4daede08ee3/tmp/profile/ptrace/BenchmarkBuildRunsc/page_cache.clean/filesystem.bindfs/benchmarks/runsc/mutex.pprof/flamegraph
PiperOrigin-RevId: 368958136
|
|
|
|
Add a coverage-report flag that will cause the sandbox to generate a coverage
report (with suffix .cov) in the debug log directory upon exiting. For the
report to be generated, runsc must have been built with the following Bazel
flags: `--collect_code_coverage --instrumentation_filter=...`.
With coverage reports, we should be able to aggregate results across all tests
to surface code coverage statistics for the project as a whole.
The report is simply a text file with each line representing a covered block
as `file:start_line.start_col,end_line.end_col`. Note that this is similar to
the format of coverage reports generated with `go test -coverprofile`,
although we omit the count and number of statements, which are not useful for
us.
Some simple ways of getting coverage reports:
bazel test <some_test> --collect_code_coverage \
--instrumentation_filter=//pkg/...
bazel build //runsc --collect_code_coverage \
--instrumentation_filter=//pkg/...
runsc -coverage-report=dir/ <other_flags> do ...
PiperOrigin-RevId: 368952911
|
|
gohacks.Memmove() takes in the number of bytes to move. The current generated
code passes len(src) and len(dst) as the number of bytes to move.
However, the marshal.Marshallable interface allows passing in larger buffers.
The stated precondition is that the buffer should be "at least" SizeBytes()
in length but it is allowed to be larger.
This change now correctly calls Memmove with the argument for the number of
bytes to move as type.SizeBytes(). This was caught when I made lisafs use the
Unsafe marshalling API and got a lot of memory violations.
PiperOrigin-RevId: 368952642
|
|
|
|
Also count failed TCP port allocations
PiperOrigin-RevId: 368939619
|
|
|
|
PiperOrigin-RevId: 368938936
|
|
|
|
PiperOrigin-RevId: 368919557
|
|
|
|
PiperOrigin-RevId: 368919504
|
|
|
|
Building nogo targets takes a very long time. This change extracts it into its
own BuildKite job.
This change also additionally speeds up other targets that were using the bazel
flag --test_tag_filters. Without --build_tag_filters, the filter is not
applied while building the specified targets and so we might end up building
targets that are not actually tested.
PiperOrigin-RevId: 368918211
|
|
|
|
PiperOrigin-RevId: 368779532
|
|
|
|
Fields allow counter metrics to have multiple tabular values.
At most one field is supported at the moment.
PiperOrigin-RevId: 368767040
|
|
|
|
PiperOrigin-RevId: 368749894
|
|
|
|
Reduce the ephemeral port range, which decreases the calls to makeEP.
PiperOrigin-RevId: 368748379
|
|
|
|
Otherwise ConnectedEndpoint.sndbuf will be restored as 0 and writes
to the socket will fail with EAGAIN.
PiperOrigin-RevId: 368746660
|
|
|
|
Thanks ianlewis@ for discovering the bug/fix!
PiperOrigin-RevId: 368740744
|
|
|
|
This was semi-automated -- there are many addresses that were not replaced.
Future commits should clean those up.
Parse4 and Parse6 were given their own package because //pkg/test can introduce
dependency cycles, as it depends transitively on //pkg/tcpip and some other
netstack packages.
PiperOrigin-RevId: 368726528
|
|
|
|
Go 1.17 is adding a new register-based calling convention [1] ("ABIInternal"),
which used is when calling between Go functions. Assembly functions are still
written using the old ABI ("ABI0"). That is, they still accept arguments on the
stack, and pass arguments to other functions on the stack. The call rules look
approximately like this:
1. Direct call from Go function to Go function: compiler emits direct
ABIInternal call.
2. Indirect call from Go function to Go function: compiler emits indirect
ABIInternal call.
3. Direct call from Go function to assembly function: compiler emits direct
ABI0 call.
4. Indirect call from Go function to assembly function: compiler emits indirect
ABIInternal call to ABI conversion wrapper function.
5. Direct or indirect call from assembly function to assembly function:
assembly/linker emits call to original ABI0 function.
6. Direct or indirect call from assembly function to Go function:
assembly/linker emits ABI0 call to ABI conversion wrapper function.
Case 4 is the interesting one here. Since the compiler can't know the ABI of an
indirect call, all indirect calls are made with ABIInternal. In order to
support indirect ABI0 assembly function calls, a wrapper is generated that
translates ABIInternal arguments to ABI0 arguments, calls the target function,
and then converts results back.
When the address of an ABI0 function is taken from Go code, it evaluates to the
address of this wrapper function rather than the target function so that later
indirect calls will work as expected.
This is normally fine, but gVisor does more than just call some of the assembly
functions we take the address of: either noting the start and end address for
future reference from a signal handler (safecopy), or copying the function text
to a new mapping (platforms).
Both of these fail with wrappers enabled (currently, this is Go tip with
GOEXPERIMENT=regabiwrappers) because these operations end up operating on the
wrapper instead of the target function.
We work around this issue by taking advantage of case 5: references to assembly
symbols from other assembly functions resolve directly to the desired target
symbol. Thus, rather than using reflect to get the address of a Go reference to
the functions, we create assembly stubs that return the address of the
function. This approach works just as well on current versions of Go, so the
change can be made immediately and doesn't require any build tags.
[1] https://go.googlesource.com/go/+/refs/heads/master/src/cmd/compile/abi-internal.md
PiperOrigin-RevId: 368505655
|
|
|
|
PiperOrigin-RevId: 368495641
|
|
|
|
PiperOrigin-RevId: 368470656
|
|
|
|
Netstack is supposed to be somewhat independent of the rest of gVisor, and
others should be able to use it without pulling in excessive dependencies.
Currently, there is no way to fight dependency creep besides careful code
review.
This change introduces a test rule `netstack_deps_check` that ensures the target
only relies on gVisor targets and a short allowlist of external dependencies.
Users who add a dependency will see an error and have to manually update the
allowlist.
The set of packages to test comes from //runsc, as it uses packages we would
expect users to commonly rely on. It was generated via:
$ find ./runsc -name BUILD | xargs grep tcpip | awk '{print $2}' | sort | uniq
(Note: We considered giving //pkg/tcpip it's own go.mod, but this breaks go
tooling.)
PiperOrigin-RevId: 368456711
|
|
|
|
We do not currently run random save tests.
PiperOrigin-RevId: 368309921
|
|
|
|
Fix a race where the ACK completing the handshake can be dropped by
a closing listener without RST to the peer. The listener close would
reset the accepted queue and that causes the connecting endpoint
in SYNRCVD state to drop the ACK thinking the queue if filled up.
PiperOrigin-RevId: 368165509
|
|
|
|
It's a common pattern in test code to reinterpret_cast<sockaddr*> from
sockaddr_* structs. Make AsSockAddr() for them so code looks better.
Note: Why not a wrapper type for `sockaddr_storage` and etc?
It's also a common need to have a local in-out variable of socklen_t.
Creating a wrapper type may however lead to this wrong code:
Wrapper addr;
socklen_t addrlen = sizeof(addr);
where sizeof(Wrapper) may not equal to sizeof(sockaddr_storage).
PiperOrigin-RevId: 368126229
|
|
|
|
PiperOrigin-RevId: 368121539
|
|
|
|
Some FileDescriptions in verity fs were opened but DecRef() were missing
after used. This could result in a ref leak.
PiperOrigin-RevId: 368096759
|
|
|
|
Reported-by: syzbot+a6ef0f95a2c9e7da26f3@syzkaller.appspotmail.com
Reported-by: syzbot+2eaf8a9f115edec468fe@syzkaller.appspotmail.com
PiperOrigin-RevId: 368093861
|
|
|
