| Age | Commit message (Collapse) | Author |
|---|
|
Remote revalidating requires to update file size on every write
on a file opened with O_APPEND. If host FD exists, it can be
used to update the size and skip round trip to the gofer. With
this change, O_APPEND writes with remote revalidating is almost
as fast as exclusive mode:
BM_Append
VFS1 60.7us
VFS2 56.8us
VFS2 exclusive 14.2us
This change 15.8us
Updates #1792
PiperOrigin-RevId: 369486801
|
|
Fixes #2926, #674
PiperOrigin-RevId: 369457123
|
|
Currently the verity action is a global variable, which causes the same
action for all verity mounts, and is overwritten for each new verity
mount. Changed it to a member of verity fs.
PiperOrigin-RevId: 369348522
|
|
While using remote-validation, the vast majority of time spent during
FS operations is re-walking the path to check for modifications and
then closing the file given that in most cases it has not been
modified externally.
This change introduces a new 9P message called MultiGetAttr which bulks
query attributes of several files in one shot. The returned attributes are
then used to update cached dentries before they are walked. File attributes
are updated for files that still exist. Dentries that have been deleted are
removed from the cache. And negative cache entries are removed if a new
file/directory was created externally. Similarly, synthetic dentries are
replaced if a file/directory is created externally.
The bulk update needs to be carefull not to follow symlinks, cross mount
points, because the gofer doesn't know how to resolve symlinks and where
mounts points are located. It also doesn't walk to the parent ("..") to
avoid deadlocks.
Here are the results:
Workload VFS1 VFS2 Change
bazel action 115s 70s 28.8s
Stat/100 11,043us 7,623us 974us
Updates #1638
PiperOrigin-RevId: 369325957
|
|
Runsc build benchmark's mutex profile shows that we are wasting roughly 25-30
seconds waiting for filesystem.renameMu to get unlocked. Earlier
checkCachingLocked required the renameMu to be locked for writing. This is a
filesystem wide lock which puts all other filesystem operations on hold and
hence is really expensive. Something to note is that all path resolution
operations hold renameMu for reading.
With this change, we allow to check for caching without even holding renameMu.
This change introduces more fine grained locks (fs.cacheMu and dentry.cachingMu)
which protect the cache (removing the requirement to hold renameMu for writing
to modify the cache) and synchronize concurrent dentry caching attempts on a per
dentry basis. We still require to hold renameMu for writing while destroying
dentries and evicting from the cache but this still significantly reduces the
write locking critical section.
Local benchmarking showed that this improved runsc build benchmark time by 4-5%.
Across 6 runs, without this change it took 310.9025 seconds to build runsc
while with this change it took 296.127 seconds.
Runsc build benchmark's mutex profile: https://gvisor.dev/profile/gvisor-buildkite/78a3f968-36ca-4944-93f7-77a8792d56b4/28a1d260-790b-4a9e-94da-a4daede08ee3/tmp/profile/ptrace/BenchmarkBuildRunsc/page_cache.clean/filesystem.bindfs/benchmarks/runsc/mutex.pprof/flamegraph
PiperOrigin-RevId: 368958136
|
|
Add a coverage-report flag that will cause the sandbox to generate a coverage
report (with suffix .cov) in the debug log directory upon exiting. For the
report to be generated, runsc must have been built with the following Bazel
flags: `--collect_code_coverage --instrumentation_filter=...`.
With coverage reports, we should be able to aggregate results across all tests
to surface code coverage statistics for the project as a whole.
The report is simply a text file with each line representing a covered block
as `file:start_line.start_col,end_line.end_col`. Note that this is similar to
the format of coverage reports generated with `go test -coverprofile`,
although we omit the count and number of statements, which are not useful for
us.
Some simple ways of getting coverage reports:
bazel test <some_test> --collect_code_coverage \
--instrumentation_filter=//pkg/...
bazel build //runsc --collect_code_coverage \
--instrumentation_filter=//pkg/...
runsc -coverage-report=dir/ <other_flags> do ...
PiperOrigin-RevId: 368952911
|
|
Otherwise ConnectedEndpoint.sndbuf will be restored as 0 and writes
to the socket will fail with EAGAIN.
PiperOrigin-RevId: 368746660
|
|
Thanks ianlewis@ for discovering the bug/fix!
PiperOrigin-RevId: 368740744
|
|
Some FileDescriptions in verity fs were opened but DecRef() were missing
after used. This could result in a ref leak.
PiperOrigin-RevId: 368096759
|
|
If the parent is not enabled in verity stepLocked(), failure to find
the child dentry could just mean an incorrect path.
PiperOrigin-RevId: 367733412
|
|
Set root dentry and root hash in verity fs before we verify the root
directory if a root hash is provided. These are used during
verification.
PiperOrigin-RevId: 367547346
|
|
We should only set parent after child is verified. Also, if the parent
is set before verified, destroyLocked() will try to grab parent.dirMu,
which may cause deadlock.
PiperOrigin-RevId: 367543655
|
|
Without this change, we ask the gofer server to update the permissions
whenever the UID, GID or size is updated via SetStat. Consequently, we don not
generate inotify events when the permissions actually change due to SGID bit
getting cleared.
With this change, we will update the permissions only when needed and generate
inotify events.
PiperOrigin-RevId: 366946842
|
|
PiperOrigin-RevId: 366923274
|
|
PiperOrigin-RevId: 366891806
|
|
Allow user mounting a verity fs on an existing mount by specifying mount
flags root_hash and lower_path.
PiperOrigin-RevId: 366843846
|
|
A skeleton implementation of cgroupfs. It supports trivial cpu and
memory controllers with no support for hierarchies.
PiperOrigin-RevId: 366561126
|
|
PiperOrigin-RevId: 366462448
|
|
Split usermem package to help remove syserror dependency in go_marshal.
New hostarch package contains code not dependent on syserror.
PiperOrigin-RevId: 365651233
|
|
On Linux these are meant to be equivalent to POLLIN/POLLOUT. Rather
than hack these on in sys_poll etc it felt cleaner to just cleanup
the call sites to notify for both events. This is what linux does
as well.
Fixes #5544
PiperOrigin-RevId: 364859977
|
|
Also adds support for clearing the setuid bit when appropriate (writing,
truncating, changing size, changing UID, or changing GID).
VFS2 only.
PiperOrigin-RevId: 364661835
|
|
PiperOrigin-RevId: 363276495
|
|
If there was a partial write (when not using the host FD) which did not generate
an error, we were incorrectly returning the number of bytes attempted to write
instead of the number of bytes actually written.
PiperOrigin-RevId: 363058989
|
|
By default, fusefs defers node permission checks to the server. The
default_permissions mount option enables the usual unix permission
checks based on the node owner and mode bits. Previously fusefs was
incorrectly checking permissions unconditionally.
Additionally, fusefs should restrict filesystem access to processes
started by the mount owner to prevent the fuse daemon from gaining
priviledge over other processes. The allow_other mount option
overrides this behaviour. Previously fusefs was incorrectly skipping
this check.
Updates #3229
PiperOrigin-RevId: 362419092
|
|
The Merkle tree files need to be cleared before enabling to avoid
redundant content.
PiperOrigin-RevId: 362409591
|
|
PiperOrigin-RevId: 362406813
|
|
Speeds up the socket stress tests by a couple orders of magnitude.
PiperOrigin-RevId: 361721050
|
|
PiperOrigin-RevId: 361196154
|
|
This validates that struct fields if annotated with "// checklocks:mu" where
"mu" is a mutex field in the same struct then access to the field is only
done with "mu" locked.
All types that are guarded by a mutex must be annotated with
// +checklocks:<mutex field name>
For more details please refer to README.md.
PiperOrigin-RevId: 360729328
|
|
The syscall package has been deprecated in favor of golang.org/x/sys.
Note that syscall is still used in the following places:
- pkg/sentry/socket/hostinet/stack.go: some netlink related functionalities
are not yet available in golang.org/x/sys.
- syscall.Stat_t is still used in some places because os.FileInfo.Sys() still
returns it and not unix.Stat_t.
Updates #214
PiperOrigin-RevId: 360701387
|
|
One precondition of VFS.PrepareRenameAt is that the `from` and `to` dentries
are not the same. Kernfs was not checking this, which could lead to a deadlock.
PiperOrigin-RevId: 359385974
|
|
Before this CL, VFS2's overlayfs uses a single private device number and an
autoincrementing generated inode number for directories; this is consistent
with Linux's overlayfs in the non-samefs non-xino case. However, this breaks
some applications more consistently than on Linux due to more aggressive
caching of Linux overlayfs dentries.
Switch from using mapped device numbers + the topmost layer's inode number for
just non-copied-up non-directory files, to doing so for all files. This still
allows directory dev/ino numbers to change across copy-up, but otherwise keeps
them consistent.
Fixes #5545:
```
$ docker run --runtime=runsc-vfs2-overlay --rm ubuntu:focal bash -c "mkdir -p 1/2/3/4/5/6/7/8 && rm -rf 1 && echo done"
done
```
PiperOrigin-RevId: 359350716
|
|
Restrict ptrace(2) according to the default configurations of the YAMA security
module (mode 1), which is a common default among various Linux distributions.
The new access checks only permit the tracer to proceed if one of the following
conditions is met:
a) The tracer is already attached to the tracee.
b) The target is a descendant of the tracer.
c) The target has explicitly given permission to the tracer through the
PR_SET_PTRACER prctl.
d) The tracer has CAP_SYS_PTRACE.
See security/yama/yama_lsm.c for more details.
Note that these checks are added to CanTrace, which is checked for
PTRACE_ATTACH as well as some other operations, e.g., checking a process'
memory layout through /proc/[pid]/mem.
Since this patch adds restrictions to ptrace, it may break compatibility for
applications run by non-root users that, for instance, rely on being able to
trace processes that are not descended from the tracer (e.g., `gdb -p`). YAMA
restrictions can be turned off by setting /proc/sys/kernel/yama/ptrace_scope
to 0, or exceptions can be made on a per-process basis with the PR_SET_PTRACER
prctl.
Reported-by: syzbot+622822d8bca08c99e8c8@syzkaller.appspotmail.com
PiperOrigin-RevId: 359237723
|
|
PiperOrigin-RevId: 357106080
|
|
PiperOrigin-RevId: 357090170
|
|
PiperOrigin-RevId: 357015186
|
|
PiperOrigin-RevId: 356868412
|
|
The limits for snd/rcv buffers for unix domain socket is controlled by the
following sysctls on linux
- net.core.rmem_default
- net.core.rmem_max
- net.core.wmem_default
- net.core.wmem_max
Today in gVisor we do not expose these sysctls but we do support setting the
equivalent in netstack via stack.Options() method. But AF_UNIX sockets in gVisor
can be used without netstack, with hostinet or even without any networking stack
at all. Which means ideally these sysctls need to live as globals in gVisor.
But rather than make this a big change for now we hardcode the limits in the
AF_UNIX implementation itself (which in itself is better than where we were
before) where it SO_SNDBUF was hardcoded to 16KiB. Further we bump the initial
limit to a default value of 208 KiB to match linux from the paltry 16 KiB we use
today.
Updates #5132
PiperOrigin-RevId: 356665498
|
|
PiperOrigin-RevId: 356450303
|
|
According to vfs.FilesystemImpl.RenameAt documentation:
- If the last path component in rp is "." or "..", and opts.Flags contains
RENAME_NOREPLACE, RenameAt returns EEXIST.
- If the last path component in rp is "." or "..", and opts.Flags does not
contain RENAME_NOREPLACE, RenameAt returns EBUSY.
Reported-by: syzbot+6189786e64fe13fe43f8@syzkaller.appspotmail.com
PiperOrigin-RevId: 355959266
|
|
PiperOrigin-RevId: 355675900
|
|
Our implementation of vfs.CheckDeleteSticky was not consistent with Linux,
specifically not consistent with fs/linux.h:check_sticky().
One of the biggest differences was that the vfs implementation did not
allow the owner of the sticky directory to delete files inside it that belonged
to other users.
This change makes our implementation consistent with Linux.
Also adds an integration test to check for this. This bug is also present in
VFS1.
Updates #3027
PiperOrigin-RevId: 355557425
|
|
When file is regular and metadata cache is authoritative, metadata lock
is taken. The code deadlocks trying to acquire the metadata lock
again to update time stampts.
PiperOrigin-RevId: 354584594
|
|
This makes it possible to add data to types that implement tcpip.Error.
ErrBadLinkEndpoint is removed as it is unused.
PiperOrigin-RevId: 354437314
|
|
Fixes #3027 as there is just 1 writable user using OrderedChildren's rename,
unlink and rmdir (kernfs.syntheticDirectory) but it doesn't support the sticky
bit yet.
Fuse which is the other writable user implements its own Inode operations.
PiperOrigin-RevId: 354386522
|
|
Contrary to the comment on the socket test, the failure was due to an issue
with goferfs rather than kernfs.
PiperOrigin-RevId: 353918021
|
|
Fixes #5113.
PiperOrigin-RevId: 353313374
|
|
PiperOrigin-RevId: 352908368
|
|
PiperOrigin-RevId: 352904728
|
|
Fixes #5263
PiperOrigin-RevId: 352903844
|
