summaryrefslogtreecommitdiffhomepage
AgeCommit message (Collapse)Author
2021-04-16[perf] Reduce contention due to renameMu in gofer client.Ayush Ranjan
Runsc build benchmark's mutex profile shows that we are wasting roughly 25-30 seconds waiting for filesystem.renameMu to get unlocked. Earlier checkCachingLocked required the renameMu to be locked for writing. This is a filesystem wide lock which puts all other filesystem operations on hold and hence is really expensive. Something to note is that all path resolution operations hold renameMu for reading. With this change, we allow to check for caching without even holding renameMu. This change introduces more fine grained locks (fs.cacheMu and dentry.cachingMu) which protect the cache (removing the requirement to hold renameMu for writing to modify the cache) and synchronize concurrent dentry caching attempts on a per dentry basis. We still require to hold renameMu for writing while destroying dentries and evicting from the cache but this still significantly reduces the write locking critical section. Local benchmarking showed that this improved runsc build benchmark time by 4-5%. Across 6 runs, without this change it took 310.9025 seconds to build runsc while with this change it took 296.127 seconds. Runsc build benchmark's mutex profile: https://gvisor.dev/profile/gvisor-buildkite/78a3f968-36ca-4944-93f7-77a8792d56b4/28a1d260-790b-4a9e-94da-a4daede08ee3/tmp/profile/ptrace/BenchmarkBuildRunsc/page_cache.clean/filesystem.bindfs/benchmarks/runsc/mutex.pprof/flamegraph PiperOrigin-RevId: 368958136
2021-04-16Allow runsc to generate coverage reports.Dean Deng
Add a coverage-report flag that will cause the sandbox to generate a coverage report (with suffix .cov) in the debug log directory upon exiting. For the report to be generated, runsc must have been built with the following Bazel flags: `--collect_code_coverage --instrumentation_filter=...`. With coverage reports, we should be able to aggregate results across all tests to surface code coverage statistics for the project as a whole. The report is simply a text file with each line representing a covered block as `file:start_line.start_col,end_line.end_col`. Note that this is similar to the format of coverage reports generated with `go test -coverprofile`, although we omit the count and number of statements, which are not useful for us. Some simple ways of getting coverage reports: bazel test <some_test> --collect_code_coverage \ --instrumentation_filter=//pkg/... bazel build //runsc --collect_code_coverage \ --instrumentation_filter=//pkg/... runsc -coverage-report=dir/ <other_flags> do ... PiperOrigin-RevId: 368952911
2021-04-16[lisa] Make go_marshal pass correctly sized buffers to safecopy.Ayush Ranjan
gohacks.Memmove() takes in the number of bytes to move. The current generated code passes len(src) and len(dst) as the number of bytes to move. However, the marshal.Marshallable interface allows passing in larger buffers. The stated precondition is that the buffer should be "at least" SizeBytes() in length but it is allowed to be larger. This change now correctly calls Memmove with the argument for the number of bytes to move as type.SizeBytes(). This was caught when I made lisafs use the Unsafe marshalling API and got a lot of memory violations. PiperOrigin-RevId: 368952642
2021-04-16Enlarge port range and fix integer overflowKevin Krakauer
Also count failed TCP port allocations PiperOrigin-RevId: 368939619
2021-04-16Include logs for packetimpact tests that are expected to failZeling Feng
PiperOrigin-RevId: 368938936
2021-04-16Use size_t instead of C integer types.Dean Deng
PiperOrigin-RevId: 368919557
2021-04-16Internal changeZach Koopmans
PiperOrigin-RevId: 368919504
2021-04-16[op] Split nogo target out of unit tests.Ayush Ranjan
Building nogo targets takes a very long time. This change extracts it into its own BuildKite job. This change also additionally speeds up other targets that were using the bazel flag --test_tag_filters. Without --build_tag_filters, the filter is not applied while building the specified targets and so we might end up building targets that are not actually tested. PiperOrigin-RevId: 368918211
2021-04-15Add verity ioctl test for mount with root hashChong Cai
PiperOrigin-RevId: 368779532
2021-04-15Add field support to the sentry metrics.Nayana Bidari
Fields allow counter metrics to have multiple tabular values. At most one field is supported at the moment. PiperOrigin-RevId: 368767040
2021-04-15Disable failing socket_ipv4_udp_unbound_loopback_test_linux tests.Dean Deng
PiperOrigin-RevId: 368749894
2021-04-15Reduce tcp_x_test runtime and memory usageKevin Krakauer
Reduce the ephemeral port range, which decreases the calls to makeEP. PiperOrigin-RevId: 368748379
2021-04-15Add S/R logic for host.ConnectedEndpointFabricio Voznika
Otherwise ConnectedEndpoint.sndbuf will be restored as 0 and writes to the socket will fail with EAGAIN. PiperOrigin-RevId: 368746660
2021-04-15Generate notification when closing host fd.Dean Deng
Thanks ianlewis@ for discovering the bug/fix! PiperOrigin-RevId: 368740744
2021-04-15Use nicer formatting for IP addresses in testsKevin Krakauer
This was semi-automated -- there are many addresses that were not replaced. Future commits should clean those up. Parse4 and Parse6 were given their own package because //pkg/test can introduce dependency cycles, as it depends transitively on //pkg/tcpip and some other netstack packages. PiperOrigin-RevId: 368726528
2021-04-14Use assembly stub to take the address of assembly functionsMichael Pratt
Go 1.17 is adding a new register-based calling convention [1] ("ABIInternal"), which used is when calling between Go functions. Assembly functions are still written using the old ABI ("ABI0"). That is, they still accept arguments on the stack, and pass arguments to other functions on the stack. The call rules look approximately like this: 1. Direct call from Go function to Go function: compiler emits direct ABIInternal call. 2. Indirect call from Go function to Go function: compiler emits indirect ABIInternal call. 3. Direct call from Go function to assembly function: compiler emits direct ABI0 call. 4. Indirect call from Go function to assembly function: compiler emits indirect ABIInternal call to ABI conversion wrapper function. 5. Direct or indirect call from assembly function to assembly function: assembly/linker emits call to original ABI0 function. 6. Direct or indirect call from assembly function to Go function: assembly/linker emits ABI0 call to ABI conversion wrapper function. Case 4 is the interesting one here. Since the compiler can't know the ABI of an indirect call, all indirect calls are made with ABIInternal. In order to support indirect ABI0 assembly function calls, a wrapper is generated that translates ABIInternal arguments to ABI0 arguments, calls the target function, and then converts results back. When the address of an ABI0 function is taken from Go code, it evaluates to the address of this wrapper function rather than the target function so that later indirect calls will work as expected. This is normally fine, but gVisor does more than just call some of the assembly functions we take the address of: either noting the start and end address for future reference from a signal handler (safecopy), or copying the function text to a new mapping (platforms). Both of these fail with wrappers enabled (currently, this is Go tip with GOEXPERIMENT=regabiwrappers) because these operations end up operating on the wrapper instead of the target function. We work around this issue by taking advantage of case 5: references to assembly symbols from other assembly functions resolve directly to the desired target symbol. Thus, rather than using reflect to get the address of a Go reference to the functions, we create assembly stubs that return the address of the function. This approach works just as well on current versions of Go, so the change can be made immediately and doesn't require any build tags. [1] https://go.googlesource.com/go/+/refs/heads/master/src/cmd/compile/abi-internal.md PiperOrigin-RevId: 368505655
2021-04-14Make the generated test binary name match the target nameTing-Yu Wang
PiperOrigin-RevId: 368495641
2021-04-14[syserror] Remove syserror from go_marshalZach Koopmans
PiperOrigin-RevId: 368470656
2021-04-14Automatically enforce limited netstack dependenciesKevin Krakauer
Netstack is supposed to be somewhat independent of the rest of gVisor, and others should be able to use it without pulling in excessive dependencies. Currently, there is no way to fight dependency creep besides careful code review. This change introduces a test rule `netstack_deps_check` that ensures the target only relies on gVisor targets and a short allowlist of external dependencies. Users who add a dependency will see an error and have to manually update the allowlist. The set of packages to test comes from //runsc, as it uses packages we would expect users to commonly rely on. It was generated via: $ find ./runsc -name BUILD | xargs grep tcpip | awk '{print $2}' | sort | uniq (Note: We considered giving //pkg/tcpip it's own go.mod, but this breaks go tooling.) PiperOrigin-RevId: 368456711
2021-04-13Remove _NoRandomSave tests.Adin Scannell
We do not currently run random save tests. PiperOrigin-RevId: 368309921
2021-04-13Fix listener close, client connect raceMithun Iyer
Fix a race where the ACK completing the handshake can be dropped by a closing listener without RST to the peer. The listener close would reset the accepted queue and that causes the connecting endpoint in SYNRCVD state to drop the ACK thinking the queue if filled up. PiperOrigin-RevId: 368165509
2021-04-12Make AsSockAddr() to replace reinterpret_cast<sockaddr*>Ting-Yu Wang
It's a common pattern in test code to reinterpret_cast<sockaddr*> from sockaddr_* structs. Make AsSockAddr() for them so code looks better. Note: Why not a wrapper type for `sockaddr_storage` and etc? It's also a common need to have a local in-out variable of socklen_t. Creating a wrapper type may however lead to this wrong code: Wrapper addr; socklen_t addrlen = sizeof(addr); where sizeof(Wrapper) may not equal to sizeof(sockaddr_storage). PiperOrigin-RevId: 368126229
2021-04-12Don't mark exported PRs as stale.Ian Lewis
PiperOrigin-RevId: 368121539
2021-04-12Add DecRef for verity FDs that were missingChong Cai
Some FileDescriptions in verity fs were opened but DecRef() were missing after used. This could result in a ref leak. PiperOrigin-RevId: 368096759
2021-04-12Don't grab TaskSet mu recursively when reading task state.Rahat Mahmood
Reported-by: syzbot+a6ef0f95a2c9e7da26f3@syzkaller.appspotmail.com Reported-by: syzbot+2eaf8a9f115edec468fe@syzkaller.appspotmail.com PiperOrigin-RevId: 368093861
2021-04-12[op] Use faster go_marshal methods in netfilter.Ayush Ranjan
Use MarshalUnsafe for packed types as it is faster than MarshalBytes. PiperOrigin-RevId: 368076368
2021-04-12Drop locks before calling waiterQueue.NotifyTamir Duberstein
Holding this lock can cause the user's callback to deadlock if it attempts to inspect the accept queue. PiperOrigin-RevId: 368068334
2021-04-12Add /etc/containerd/runsc.toml to conffiles attribute.Adin Scannell
Fixes #5817 PiperOrigin-RevId: 368060056
2021-04-10Use the SecureRNG to generate listener noncesTamir Duberstein
Some other cleanup while I'm here: - Remove unused arguments - Handle some unhandled errors - Remove redundant casts - Remove redundant parens - Avoid shadowing `hash` package name PiperOrigin-RevId: 367816161
2021-04-10Don't store accepted endpoints in a channelTamir Duberstein
Use a linked list with cached length and capacity. The current channel is already composed with a mutex and condition variable, and is never used for its channel-like properties. Channels also require eager allocation equal to their capacity, which a linked list does not. PiperOrigin-RevId: 367766626
2021-04-09iptables: support postrouting hook and SNAT targetToshi Kikuchi
The current SNAT implementation has several limitations: - SNAT source port has to be specified. It is not optional. - SNAT source port range is not supported. - SNAT for UDP is a one-way translation. No response packets are handled (because conntrack doesn't support UDP currently). - SNAT and REDIRECT can't work on the same connection. Fixes #5489 PiperOrigin-RevId: 367750325
2021-04-09Return integrity failure only if enabledChong Cai
If the parent is not enabled in verity stepLocked(), failure to find the child dentry could just mean an incorrect path. PiperOrigin-RevId: 367733412
2021-04-09Merge pull request #5767 from avagin:mxcsrgVisor bot
PiperOrigin-RevId: 367730917
2021-04-09Move maxListenBacklog check to sentryMithun Iyer
Move maxListenBacklog check to the caller of endpoint Listen so that it is applicable to Unix domain sockets as well. This was changed in cl/366935921. Reported-by: syzbot+a35ae7cdfdde0c41cf7a@syzkaller.appspotmail.com PiperOrigin-RevId: 367728052
2021-04-09Rename IsV6LinkLocalAddress to IsV6LinkLocalUnicastAddressGhanan Gowripalan
To match the V4 variant. PiperOrigin-RevId: 367691981
2021-04-09Remove duplicate accept queue fullness checkTamir Duberstein
Both code paths perform this check; extract it and remove the comment that suggests it is unique to one of the paths. PiperOrigin-RevId: 367666160
2021-04-09Propagate SYN handling errorTamir Duberstein
Both callers of this function still drop this error on the floor, but progress is progress. Updates #4690. PiperOrigin-RevId: 367604788
2021-04-08Set root dentry and hash for verity before verifyChong Cai
Set root dentry and root hash in verity fs before we verify the root directory if a root hash is provided. These are used during verification. PiperOrigin-RevId: 367547346
2021-04-08Set parent after child is verifiedChong Cai
We should only set parent after child is verified. Also, if the parent is set before verified, destroyLocked() will try to grab parent.dirMu, which may cause deadlock. PiperOrigin-RevId: 367543655
2021-04-08Merge pull request #5736 from lubinszARM:pr_bblu_tlb_asidgVisor bot
PiperOrigin-RevId: 367523491
2021-04-08Do not forward link-local packetsGhanan Gowripalan
As per RFC 3927 section 7 and RFC 4291 section 2.5.6. Test: forward_test.TestMulticastForwarding PiperOrigin-RevId: 367519336
2021-04-08Drop unused escapes information.Adin Scannell
PiperOrigin-RevId: 367517305
2021-04-08Add Children in merkletree generateChong Cai
This field was missing and should be provided. PiperOrigin-RevId: 367474481
2021-04-08Join all routers group when forwarding is enabledGhanan Gowripalan
See comments inline code for rationale. Test: ip_test.TestJoinLeaveAllRoutersGroup PiperOrigin-RevId: 367449434
2021-04-08Clarify platform errors.Adin Scannell
PiperOrigin-RevId: 367446222
2021-04-07Add internal staging tags to //runsc and //shim binaries.Adin Scannell
PiperOrigin-RevId: 367328273
2021-04-07perf/getpid: add a case when syscalls are executed via mov $XXX, %eax; syscallAndrei Vagin
This is the most often pattern of calling system calls in real applications. PiperOrigin-RevId: 367320048
2021-04-07Remove flock suppressionTamir Duberstein
PiperOrigin-RevId: 367312275
2021-04-06Do not perform MLD for certain multicast scopesGhanan Gowripalan
...as per RFC 2710 section 5 page 10. Test: ipv6_test.TestMLDSkipProtocol PiperOrigin-RevId: 367031126
2021-04-05Update gofer dentry permissions only when needed.Ayush Ranjan
Without this change, we ask the gofer server to update the permissions whenever the UID, GID or size is updated via SetStat. Consequently, we don not generate inotify events when the permissions actually change due to SGID bit getting cleared. With this change, we will update the permissions only when needed and generate inotify events. PiperOrigin-RevId: 366946842
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-18 16:07:17 +0000