summaryrefslogtreecommitdiffhomepage
path: root/pkg/sentry/fsimpl
AgeCommit message (Collapse)Author
2021-06-29Sort children map before hashChong Cai
The unordered map may generate different hash due to its order. The children map needs to be sorted each time before hashing to avoid false verification failure due to the map. Store the sorted children map in verity dentry to avoid sorting it each time verification happens. Also serialize the whole VerityDescriptor struct to hash now that the map is removed from it. PiperOrigin-RevId: 382201560
2021-06-29[syserror] Change syserror to linuxerr for E2BIG, EADDRINUSE, and EINVALZach Koopmans
Remove three syserror entries duplicated in linuxerr. Because of the linuxerr.Equals method, this is a mere change of return values from syserror to linuxerr definitions. Done with only these three errnos as CLs removing all grow to a significantly large size. PiperOrigin-RevId: 382173835
2021-06-28Allow VFS2 gofer client to mmap from sentry page cache when forced.Jamie Liu
PiperOrigin-RevId: 381982257
2021-06-24Internal change.Jamie Liu
PiperOrigin-RevId: 381375705
2021-06-22[syserror] Add conversions to linuxerr with temporary Equals method.Zach Koopmans
Add Equals method to compare syserror and unix.Errno errors to linuxerr errors. This will facilitate removal of syserror definitions in a followup, and finding needed conversions from unix.Errno to linuxerr. PiperOrigin-RevId: 380909667
2021-06-17Move tcpip.Clock impl to TimekeeperTamir Duberstein
...and pass it explicitly. This reverts commit b63e61828d0652ad1769db342c17a3529d2d24ed. PiperOrigin-RevId: 380039167
2021-06-10Minor VFS2 xattr changes.Jamie Liu
- Allow the gofer client to use most xattr namespaces. As documented by the updated comment, this is consistent with e.g. Linux's FUSE client, and allows gofers to provide extended attributes from FUSE filesystems. - Make tmpfs' listxattr omit xattrs in the "trusted" namespace for non-privileged users. PiperOrigin-RevId: 378778854
2021-06-10Fix lock ordering issue when enumerating cgroup tasks.Rahat Mahmood
The control files enumerating tasks and threads residing in cgroupfs incorrectly locks cgroupfs.filesystem.tasksMu before kernel.TaskSet.mu. The contents of these control files are inherently racy anyways, so use a snapshot of the tasks in the cgroup and drop tasksMu before resolving pids/tids (which acquires TaskSet.mu). PiperOrigin-RevId: 378767060
2021-06-10Add /proc/sys/vm/max_map_countFabricio Voznika
Set it to int32 max because gVisor doesn't have a limit. Fixes #2337 PiperOrigin-RevId: 378722230
2021-06-07Implement RENAME_NOREPLACE for all VFS2 filesystem implementations.Jamie Liu
PiperOrigin-RevId: 377966969
2021-06-03Initialize metrics at initTamir Duberstein
Avoids a race condition at kernel initialization. Updates #6057. PiperOrigin-RevId: 377357723
2021-05-26Add verity getdents testsChong Cai
PiperOrigin-RevId: 376001603
2021-05-25Initialize Kernel.Timekeeper before network NSTamir Duberstein
PiperOrigin-RevId: 375843579
2021-05-25Use specific fmt verbs (avoid %v)Tamir Duberstein
Remove useless conversions. Avoid unhandled errors. PiperOrigin-RevId: 375834275
2021-05-25setgid directories for VFS1 tmpfs, overlayfs, and goferfsKevin Krakauer
PiperOrigin-RevId: 375780659
2021-05-18Delete /cloud/gvisor/sandbox/sentry/gofer/opened_write_execute_file metricNayana Bidari
This metric is replaced by /cloud/gvisor/sandbox/sentry/suspicious_operations metric with field value opened_write_execute_file. PiperOrigin-RevId: 374509823
2021-05-14Add new metric for suspicious operations.Nayana Bidari
The new metric contains fields and will replace the below existing metric: - opened_write_execute_file PiperOrigin-RevId: 373884604
2021-05-14Resolve remaining O_PATH TODOs.Dean Deng
O_PATH is now implemented in vfs2. Fixes #2782. PiperOrigin-RevId: 373861410
2021-05-14Don't read forwarding from netstack in sentryGhanan Gowripalan
https://www.kernel.org/doc/Documentation/networking/ip-sysctl.txt: /proc/sys/net/ipv4/* Variables: ip_forward - BOOLEAN 0 - disabled (default) not 0 - enabled Forward Packets between interfaces. This variable is special, its change resets all configuration parameters to their default state (RFC1122 for hosts, RFC1812 for routers) /proc/sys/net/ipv4/ip_forward only does work when its value is changed and always returns the last written value. The last written value may not reflect the current state of the netstack (e.g. when `ip_forward` was written a value of "1" then disable forwarding on an interface) so there is no need for sentry to probe netstack to get the current forwarding state of interfaces. ``` ~$ cat /proc/sys/net/ipv4/ip_forward 0 ~$ sudo bash -c "echo 1 > /proc/sys/net/ipv4/ip_forward" ~$ cat /proc/sys/net/ipv4/ip_forward 1 ~$ sudo sysctl -a | grep ipv4 | grep forward net.ipv4.conf.all.forwarding = 1 net.ipv4.conf.default.forwarding = 1 net.ipv4.conf.eno1.forwarding = 1 net.ipv4.conf.lo.forwarding = 1 net.ipv4.conf.wlp1s0.forwarding = 1 net.ipv4.ip_forward = 1 net.ipv4.ip_forward_update_priority = 1 net.ipv4.ip_forward_use_pmtu = 0 ~$ sudo sysctl -w net.ipv4.conf.wlp1s0.forwarding=0 net.ipv4.conf.wlp1s0.forwarding = 0 ~$ sudo sysctl -a | grep ipv4 | grep forward net.ipv4.conf.all.forwarding = 1 net.ipv4.conf.default.forwarding = 1 net.ipv4.conf.eno1.forwarding = 1 net.ipv4.conf.lo.forwarding = 1 net.ipv4.conf.wlp1s0.forwarding = 0 net.ipv4.ip_forward = 1 net.ipv4.ip_forward_update_priority = 1 net.ipv4.ip_forward_use_pmtu = 0 ~$ cat /proc/sys/net/ipv4/ip_forward 1 ~$ sudo bash -c "echo 1 > /proc/sys/net/ipv4/ip_forward" ~$ sudo sysctl -a | grep ipv4 | grep forward net.ipv4.conf.all.forwarding = 1 net.ipv4.conf.default.forwarding = 1 net.ipv4.conf.eno1.forwarding = 1 net.ipv4.conf.lo.forwarding = 1 net.ipv4.conf.wlp1s0.forwarding = 0 net.ipv4.ip_forward = 1 net.ipv4.ip_forward_update_priority = 1 net.ipv4.ip_forward_use_pmtu = 0 ~$ sudo bash -c "echo 0 > /proc/sys/net/ipv4/ip_forward" ~$ sudo sysctl -a | grep ipv4 | grep forward sysctl: unable to open directory "/proc/sys/fs/binfmt_misc/" net.ipv4.conf.all.forwarding = 0 net.ipv4.conf.default.forwarding = 0 net.ipv4.conf.eno1.forwarding = 0 net.ipv4.conf.lo.forwarding = 0 net.ipv4.conf.wlp1s0.forwarding = 0 net.ipv4.ip_forward = 0 net.ipv4.ip_forward_update_priority = 1 net.ipv4.ip_forward_use_pmtu = 0 ~$ cat /proc/sys/net/ipv4/ip_forward 0 ``` In the above example we can see that writing "1" to /proc/sys/net/ipv4/ip_forward configures the stack to be a router (all interfaces are configured to enable forwarding). However, if we manually update an interace (`wlp1s0`) to not forward packets, /proc/sys/net/ipv4/ip_forward continues to return the last written value of "1", even though not all interfaces will forward packets. Also note that writing the same value twice has no effect; work is performed iff the value changes. This change also removes the 'unset' state from sentry's ip forwarding data structures as an 'unset' ip forwarding value is the same as leaving forwarding disabled as the stack is always brought up with forwarding initially disabled; disabling forwarding on a newly created stack is a no-op. PiperOrigin-RevId: 373853106
2021-05-14Fix cgroup hierarchy registration.Rahat Mahmood
Previously, registration was racy because we were publishing hierarchies in the registry without fully initializing the underlying filesystem. This led to concurrent mount(2)s discovering the partially intialized filesystems and dropping the final refs on them which cause them to be freed prematurely. Reported-by: syzbot+13f54e77bdf59f0171f0@syzkaller.appspotmail.com Reported-by: syzbot+2c7f0a9127ac6a84f17e@syzkaller.appspotmail.com PiperOrigin-RevId: 373824552
2021-05-12Fix TODO comments.Ian Lewis
Fix TODO comments referring to incorrect issue numbers. Also fix the link in issue reviver comments to include the right url fragment. PiperOrigin-RevId: 373491821
2021-05-06Implement /proc/cmdlineSteve Silva
This change implements /proc/cmdline with a basic faux command line "BOOT_IMAGE=/vmlinuz-[version]-gvisor quiet" so apps that may expect it do not receive errors. Also tests for the existence of /proc/cmdline as part of the system call test suite PiperOrigin-RevId: 372462070
2021-05-04Remove uses of the binary package from the rest of the sentry.Rahat Mahmood
PiperOrigin-RevId: 372020696
2021-05-03Fix deadlock in /proc/[pid]/fd/[num]Fabricio Voznika
In order to resolve path names, fsSymlink.Readlink() may need to reenter kernfs. Change the code so that kernfs.Inode.Readlink() is called without locks and document the new contract. PiperOrigin-RevId: 371770222
2021-05-01[perf] Check caching on IncRef'd dentries before the others.Ayush Ranjan
When a child is added to a parent (directory) dentry, both child and parent are queued for checkCachingLocked(). Make sure that the parent is queued first because the parent gained a ref and so could be removed from the LRU cache hence making space for the new child. This could prevent an LRU cache eviction. In practice, this did seem to help. ~800 RPCs were reduced while building //absl/... (ABSL build benchmark). Evictions hurt in 2 ways - create renameMu contention and destroy a possibly useful dentry which will have to be re-walked and re-opened later. Follow up fix for #5859. PiperOrigin-RevId: 371509392
2021-04-30Do not return content if verity translate failsChong Cai
If verification fails for translating mmapped memory, the content should not be returned. This is not an issue for panic mode, but for error mode we should return empty content along with the error. PiperOrigin-RevId: 371393519
2021-04-29[perf] Remove unnecessary existence checks in doCreateAt().Ayush Ranjan
Originally we were making a WalkGetAttrOne RPC to confirm that a file does not exist on the remote filesystem - when there was no cached information about the existence of a dentry at that position. This change avoids making that RPC and speculatively makes the mkdir/mknod/linkat/symlink RPC. They will fail with EEXIST if a file exists at that position as we want. However the error ordering is important. Existence check comes before writability check. So we make the existence check when the writability check fails and give it precedence. This change saves ~76,000 RPCs while building //absl/... (ABSL build benchmark). That is 10% of all RPCs made while running that workload. PiperOrigin-RevId: 371225633
2021-04-29Remove outdated TODOs in verityChong Cai
PiperOrigin-RevId: 371198372
2021-04-28Automated rollback of changelist 369686285Fabricio Voznika
PiperOrigin-RevId: 371015541
2021-04-22Add weirdness sentry metric.Nayana Bidari
Weirdness metric contains fields to track the number of clock fallback, partial result and vsyscalls. This metric will avoid the overhead of having three different metrics (fallbackMetric, partialResultMetric, vsyscallCount). PiperOrigin-RevId: 369970218
2021-04-21Stub the custom "job" controller required by some workloads.Rahat Mahmood
PiperOrigin-RevId: 369724358
2021-04-21Automated rollback of changelist 369325957Michael Pratt
PiperOrigin-RevId: 369686285
2021-04-20[perf] Remove non-empty directory dentries from gofer LRU cache.Ayush Ranjan
The gofer client's LRU cache has a default limit of 1000 dentries. Any attempt to cache more dentries than that will make the LRU cache evict and destroy the least recently used dentry. However, the eviction is expensive because it requires holding fs.renameMu for writing - which in turn creates a lot of contention. All filesystem operations that involve path traversal require fs.renameMu for reading atleast. Therefore, it is in our best interest to keep the cache small and clean. When a dentry is inserted in the dentry tree, it grabs a ref on its parent for its entire lifetime. Hence the parent is longer evictable (because refs > 0). This change additionally calls checkCachingLocked on directories that have been added to so that they can be removed from the LRU cache if needed. This change implies that the LRU cache will only contain the leaves from the filesystem tree which significantly reduces the LRU cache size and consequently reduces the number of expensive LRU cache evictions. > Why are opened dentries not removed from LRU cache? When a file description is open(2)-ed, the file description holds a ref on its dentry for its entire lifetime. However, calling checkCachingLocked() on opened dentries actually ends up hurting performance. Applications usually open file descriptors for a short duration. So upon close(2), the dentry is reinserted into the cache anyway. So the precautionary work done in removing the opened dentry from the cache went for waste as it did not really reduce an eviction. Local benchmarking has shown that this change improves performance by 3-4%. Across 6 runs, without this change it took 296.127 seconds to build runsc while with this change it took only 285.136 seconds. PiperOrigin-RevId: 369510494
2021-04-20Speed up O_APPEND with remote revalidatingFabricio Voznika
Remote revalidating requires to update file size on every write on a file opened with O_APPEND. If host FD exists, it can be used to update the size and skip round trip to the gofer. With this change, O_APPEND writes with remote revalidating is almost as fast as exclusive mode: BM_Append VFS1 60.7us VFS2 56.8us VFS2 exclusive 14.2us This change 15.8us Updates #1792 PiperOrigin-RevId: 369486801
2021-04-20Move SO_RCVBUF to socketops.Nayana Bidari
Fixes #2926, #674 PiperOrigin-RevId: 369457123
2021-04-19Change verity action to be a fs memberChong Cai
Currently the verity action is a global variable, which causes the same action for all verity mounts, and is overwritten for each new verity mount. Changed it to a member of verity fs. PiperOrigin-RevId: 369348522
2021-04-19Add MultiGetAttr message to 9PFabricio Voznika
While using remote-validation, the vast majority of time spent during FS operations is re-walking the path to check for modifications and then closing the file given that in most cases it has not been modified externally. This change introduces a new 9P message called MultiGetAttr which bulks query attributes of several files in one shot. The returned attributes are then used to update cached dentries before they are walked. File attributes are updated for files that still exist. Dentries that have been deleted are removed from the cache. And negative cache entries are removed if a new file/directory was created externally. Similarly, synthetic dentries are replaced if a file/directory is created externally. The bulk update needs to be carefull not to follow symlinks, cross mount points, because the gofer doesn't know how to resolve symlinks and where mounts points are located. It also doesn't walk to the parent ("..") to avoid deadlocks. Here are the results: Workload VFS1 VFS2 Change bazel action 115s 70s 28.8s Stat/100 11,043us 7,623us 974us Updates #1638 PiperOrigin-RevId: 369325957
2021-04-16[perf] Reduce contention due to renameMu in gofer client.Ayush Ranjan
Runsc build benchmark's mutex profile shows that we are wasting roughly 25-30 seconds waiting for filesystem.renameMu to get unlocked. Earlier checkCachingLocked required the renameMu to be locked for writing. This is a filesystem wide lock which puts all other filesystem operations on hold and hence is really expensive. Something to note is that all path resolution operations hold renameMu for reading. With this change, we allow to check for caching without even holding renameMu. This change introduces more fine grained locks (fs.cacheMu and dentry.cachingMu) which protect the cache (removing the requirement to hold renameMu for writing to modify the cache) and synchronize concurrent dentry caching attempts on a per dentry basis. We still require to hold renameMu for writing while destroying dentries and evicting from the cache but this still significantly reduces the write locking critical section. Local benchmarking showed that this improved runsc build benchmark time by 4-5%. Across 6 runs, without this change it took 310.9025 seconds to build runsc while with this change it took 296.127 seconds. Runsc build benchmark's mutex profile: https://gvisor.dev/profile/gvisor-buildkite/78a3f968-36ca-4944-93f7-77a8792d56b4/28a1d260-790b-4a9e-94da-a4daede08ee3/tmp/profile/ptrace/BenchmarkBuildRunsc/page_cache.clean/filesystem.bindfs/benchmarks/runsc/mutex.pprof/flamegraph PiperOrigin-RevId: 368958136
2021-04-16Allow runsc to generate coverage reports.Dean Deng
Add a coverage-report flag that will cause the sandbox to generate a coverage report (with suffix .cov) in the debug log directory upon exiting. For the report to be generated, runsc must have been built with the following Bazel flags: `--collect_code_coverage --instrumentation_filter=...`. With coverage reports, we should be able to aggregate results across all tests to surface code coverage statistics for the project as a whole. The report is simply a text file with each line representing a covered block as `file:start_line.start_col,end_line.end_col`. Note that this is similar to the format of coverage reports generated with `go test -coverprofile`, although we omit the count and number of statements, which are not useful for us. Some simple ways of getting coverage reports: bazel test <some_test> --collect_code_coverage \ --instrumentation_filter=//pkg/... bazel build //runsc --collect_code_coverage \ --instrumentation_filter=//pkg/... runsc -coverage-report=dir/ <other_flags> do ... PiperOrigin-RevId: 368952911
2021-04-15Add S/R logic for host.ConnectedEndpointFabricio Voznika
Otherwise ConnectedEndpoint.sndbuf will be restored as 0 and writes to the socket will fail with EAGAIN. PiperOrigin-RevId: 368746660
2021-04-15Generate notification when closing host fd.Dean Deng
Thanks ianlewis@ for discovering the bug/fix! PiperOrigin-RevId: 368740744
2021-04-12Add DecRef for verity FDs that were missingChong Cai
Some FileDescriptions in verity fs were opened but DecRef() were missing after used. This could result in a ref leak. PiperOrigin-RevId: 368096759
2021-04-09Return integrity failure only if enabledChong Cai
If the parent is not enabled in verity stepLocked(), failure to find the child dentry could just mean an incorrect path. PiperOrigin-RevId: 367733412
2021-04-08Set root dentry and hash for verity before verifyChong Cai
Set root dentry and root hash in verity fs before we verify the root directory if a root hash is provided. These are used during verification. PiperOrigin-RevId: 367547346
2021-04-08Set parent after child is verifiedChong Cai
We should only set parent after child is verified. Also, if the parent is set before verified, destroyLocked() will try to grab parent.dirMu, which may cause deadlock. PiperOrigin-RevId: 367543655
2021-04-05Update gofer dentry permissions only when needed.Ayush Ranjan
Without this change, we ask the gofer server to update the permissions whenever the UID, GID or size is updated via SetStat. Consequently, we don not generate inotify events when the permissions actually change due to SGID bit getting cleared. With this change, we will update the permissions only when needed and generate inotify events. PiperOrigin-RevId: 366946842
2021-04-05Report task CPU usage through the cpuacct cgroup controller.Rahat Mahmood
PiperOrigin-RevId: 366923274
2021-04-05Allow default control values to be set for cgroupfs.Rahat Mahmood
PiperOrigin-RevId: 366891806
2021-04-05Allow user mount for verity fsChong Cai
Allow user mounting a verity fs on an existing mount by specifying mount flags root_hash and lower_path. PiperOrigin-RevId: 366843846
2021-04-02Implement cgroupfs.Rahat Mahmood
A skeleton implementation of cgroupfs. It supports trivial cpu and memory controllers with no support for hierarchies. PiperOrigin-RevId: 366561126
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-02 08:29:02 +0000