summaryrefslogtreecommitdiffhomepage
path: root/pkg/sentry/vfs
AgeCommit message (Collapse)Author
2020-12-09Add //pkg/sync:generic_atomicptrmap.Jamie Liu
AtomicPtrMap is a generic concurrent map from arbitrary keys to arbitrary pointer values. Benchmarks: name time/op StoreDelete/RWMutexMap-12 335ns ± 1% StoreDelete/SyncMap-12 705ns ± 3% StoreDelete/AtomicPtrMap-12 287ns ± 4% StoreDelete/AtomicPtrMapSharded-12 289ns ± 1% LoadOrStoreDelete/RWMutexMap-12 342ns ± 2% LoadOrStoreDelete/SyncMap-12 662ns ± 2% LoadOrStoreDelete/AtomicPtrMap-12 290ns ± 7% LoadOrStoreDelete/AtomicPtrMapSharded-12 293ns ± 2% LookupPositive/RWMutexMap-12 101ns ±26% LookupPositive/SyncMap-12 202ns ± 2% LookupPositive/AtomicPtrMap-12 71.1ns ± 2% LookupPositive/AtomicPtrMapSharded-12 73.2ns ± 1% LookupNegative/RWMutexMap-12 119ns ± 1% LookupNegative/SyncMap-12 154ns ± 1% LookupNegative/AtomicPtrMap-12 84.7ns ± 3% LookupNegative/AtomicPtrMapSharded-12 86.8ns ± 1% Concurrent/FixedKeys_1PercentWrites_RWMutexMap-12 1.32µs ± 2% Concurrent/FixedKeys_1PercentWrites_SyncMap-12 52.7ns ±10% Concurrent/FixedKeys_1PercentWrites_AtomicPtrMap-12 31.8ns ±20% Concurrent/FixedKeys_1PercentWrites_AtomicPtrMapSharded-12 24.0ns ±15% Concurrent/FixedKeys_10PercentWrites_RWMutexMap-12 860ns ± 3% Concurrent/FixedKeys_10PercentWrites_SyncMap-12 68.8ns ±20% Concurrent/FixedKeys_10PercentWrites_AtomicPtrMap-12 98.6ns ± 7% Concurrent/FixedKeys_10PercentWrites_AtomicPtrMapSharded-12 42.0ns ±25% Concurrent/FixedKeys_50PercentWrites_RWMutexMap-12 1.17µs ± 3% Concurrent/FixedKeys_50PercentWrites_SyncMap-12 136ns ±34% Concurrent/FixedKeys_50PercentWrites_AtomicPtrMap-12 286ns ± 3% Concurrent/FixedKeys_50PercentWrites_AtomicPtrMapSharded-12 115ns ±35% Concurrent/ChangingKeys_1PercentWrites_RWMutexMap-12 1.27µs ± 2% Concurrent/ChangingKeys_1PercentWrites_SyncMap-12 5.01µs ± 3% Concurrent/ChangingKeys_1PercentWrites_AtomicPtrMap-12 38.1ns ± 3% Concurrent/ChangingKeys_1PercentWrites_AtomicPtrMapSharded-12 22.6ns ± 2% Concurrent/ChangingKeys_10PercentWrites_RWMutexMap-12 1.08µs ± 2% Concurrent/ChangingKeys_10PercentWrites_SyncMap-12 5.97µs ± 1% Concurrent/ChangingKeys_10PercentWrites_AtomicPtrMap-12 390ns ± 2% Concurrent/ChangingKeys_10PercentWrites_AtomicPtrMapSharded-12 93.6ns ± 1% Concurrent/ChangingKeys_50PercentWrites_RWMutexMap-12 1.77µs ± 2% Concurrent/ChangingKeys_50PercentWrites_SyncMap-12 8.07µs ± 2% Concurrent/ChangingKeys_50PercentWrites_AtomicPtrMap-12 1.61µs ± 2% Concurrent/ChangingKeys_50PercentWrites_AtomicPtrMapSharded-12 386ns ± 1% Updates #231 PiperOrigin-RevId: 346614776
2020-12-04Overlay runsc regular file mounts with regular files.Jamie Liu
Fixes #4991 PiperOrigin-RevId: 345800333
2020-12-03Implement `fcntl` options `F_GETSIG` and `F_SETSIG`.Etienne Perot
These options allow overriding the signal that gets sent to the process when I/O operations are available on the file descriptor, rather than the default `SIGIO` signal. Doing so also populates `siginfo` to contain extra information about which file descriptor caused the event (`si_fd`) and what events happened on it (`si_band`). The logic around which FD is populated within `si_fd` matches Linux's, which means it has some weird edge cases where that value may not actually refer to a file descriptor that is still valid. This CL also ports extra S/R logic regarding async handler in VFS2. Without this, async I/O handlers aren't properly re-registered after S/R. PiperOrigin-RevId: 345436598
2020-12-02Remove FileReadWriteSeeker from vfs.Jamie Liu
Previous experience has shown that these types of wrappers tends to create two kinds of problems: hidden allocations (e.g. each call to FileReadWriteSeeker.Read/Write allocates a usermem.BytesIO on the heap) and hidden lock ordering problems (e.g. VFS1 splice deadlocks). Since this is only needed by fsimpl/verity, move it there. PiperOrigin-RevId: 345377830
2020-11-18Port filesystem metrics to VFS2.Jamie Liu
PiperOrigin-RevId: 343196927
2020-11-09Initialize references with a value of 1.Dean Deng
This lets us avoid treating a value of 0 as one reference. All references using the refsvfs2 template must call InitRefs() before the reference is incremented/decremented, or else a panic will occur. Therefore, it should be pretty easy to identify missing InitRef calls during testing. Updates #1486. PiperOrigin-RevId: 341411151
2020-11-03Fix more nogo testsTing-Yu Wang
PiperOrigin-RevId: 340536306
2020-11-03[vfs2] Do not drop inotify waiters across S/R.Dean Deng
The waits-for relationship between an epoll instance and an inotify fd should be restored. This fixes flaky inotify vfs2 tests. PiperOrigin-RevId: 340531367
2020-10-28Add logging option to leak checker.Dean Deng
Also refactor the template and CheckedObject interface to make this cleaner. Updates #1486. PiperOrigin-RevId: 339577120
2020-10-24Add leak checking to vfs2 structures that cannot use the refs_vfs2 template.Dean Deng
Updates #1486. PiperOrigin-RevId: 338832085
2020-10-23Support VFS2 save/restore.Jamie Liu
Inode number consistency checks are now skipped in save/restore tests for reasons described in greatest detail in StatTest.StateDoesntChangeAfterRename. They pass in VFS1 due to the bug described in new test case SimpleStatTest.DifferentFilesHaveDifferentDeviceInodeNumberPairs. Fixes #1663 PiperOrigin-RevId: 338776148
2020-10-23Rewrite reference leak checker without finalizers.Dean Deng
Our current reference leak checker uses finalizers to verify whether an object has reached zero references before it is garbage collected. There are multiple problems with this mechanism, so a rewrite is in order. With finalizers, there is no way to guarantee that a finalizer will run before the program exits. When an unreachable object with a finalizer is garbage collected, its finalizer will be added to a queue and run asynchronously. The best we can do is run garbage collection upon sandbox exit to make sure that all finalizers are enqueued. Furthermore, if there is a chain of finalized objects, e.g. A points to B points to C, garbage collection needs to run multiple times before all of the finalizers are enqueued. The first GC run will register the finalizer for A but not free it. It takes another GC run to free A, at which point B's finalizer can be registered. As a result, we need to run GC as many times as the length of the longest such chain to have a somewhat reliable leak checker. Finally, a cyclical chain of structs pointing to one another will never be garbage collected if a finalizer is set. This is a well-known issue with Go finalizers (https://github.com/golang/go/issues/7358). Using leak checking on filesystem objects that produce cycles will not work and even result in memory leaks. The new leak checker stores reference counted objects in a global map when leak check is enabled and removes them once they are destroyed. At sandbox exit, any remaining objects in the map are considered as leaked. This provides a deterministic way of detecting leaks without relying on the complexities of finalizers and garbage collection. This approach has several benefits over the former, including: - Always detects leaks of objects that should be destroyed very close to sandbox exit. The old checker very rarely detected these leaks, because it relied on garbage collection to be run in a short window of time. - Panics if we forgot to enable leak check on a ref-counted object (we will try to remove it from the map when it is destroyed, but it will never have been added). - Can store extra logging information in the map values without adding to the size of the ref count struct itself. With the size of just an int64, the ref count object remains compact, meaning frequent operations like IncRef/DecRef are more cache-efficient. - Can aggregate leak results in a single report after the sandbox exits. Instead of having warnings littered in the log, which were non-deterministically triggered by garbage collection, we can print all warning messages at once. Note that this could also be a limitation--the sandbox must exit properly for leaks to be detected. Some basic benchmarking indicates that this change does not significantly affect performance when leak checking is enabled, which is understandable since registering/unregistering is only done once for each filesystem object. Updates #1486. PiperOrigin-RevId: 338685972
2020-10-13[vfs2] Don't take reference in Task.MountNamespaceVFS2 and MountNamespace.Root.Dean Deng
This fixes reference leaks related to accidentally forgetting to DecRef() after calling one or the other. PiperOrigin-RevId: 336918922
2020-10-13[vfs2] Add FilesystemType.Release to avoid reference leaks.Dean Deng
Singleton filesystem like devpts and devtmpfs have a single filesystem shared among all mounts, so they acquire a "self-reference" when initialized that must be released when the entire virtual filesystem is released at sandbox exit. PiperOrigin-RevId: 336828852
2020-10-12Change Merkle tree library to use ReaderAtChong Cai
Merkle tree library was originally using Read/Seek to access data and tree, since the parameters are io.ReadSeeker. This could cause race conditions if multiple threads accesses the same fd to read. Here we change to use ReaderAt, and implement it with PRead to make it thread safe. PiperOrigin-RevId: 336779260
2020-10-12[vfs2] Don't leak disconnected mounts.Dean Deng
PiperOrigin-RevId: 336694658
2020-10-02Convert uses of the binary package in kernel to go-marshal.Rahat Mahmood
PiperOrigin-RevId: 335077195
2020-09-24Add basic stateify annotations.Adin Scannell
Updates #1663 PiperOrigin-RevId: 333539293
2020-09-23Add more descriptive comments on mount options.Dean Deng
PiperOrigin-RevId: 333447255
2020-09-17Complete vfs2 implementation of fallocate.Dean Deng
This change includes overlay, special regular gofer files, and hostfs. Fixes #3589. PiperOrigin-RevId: 332330860
2020-09-15Read vfs2 epoll events atomically.Jamie Liu
Discovered by ayushranjan@: VFS2 was employing the following algorithm for fetching ready events from an epoll instance: - Create a statically sized EpollEvent slice on the stack of size 16. - Pass that to EpollInstance.ReadEvents() to populate. - EpollInstance.ReadEvents() requeues level-triggered events that it returns back into the ready queue. - Write the results to usermem. - If the number of results were = 16 then recall EpollInstance.ReadEvents() in the hopes of getting more. But this will cause duplication of the "requeued" ready level-triggered events. So if the ready queue has >= 16 ready events, the EpollWait for loop will spin until it fills the usermem with `maxEvents` events. Fixes #3521 PiperOrigin-RevId: 331840527
2020-09-09Unlock VFS.mountMu before FilesystemImpl calls for ↵Jamie Liu
/proc/[pid]/{mounts,mountinfo}. Also move VFS.MakeSyntheticMountpoint() (which is a utility wrapper around VFS.MkdirAllAt(), itself a utility wrapper around VFS.MkdirAt()) to not be in the middle of the implementation of these proc files. Fixes #3878 PiperOrigin-RevId: 330843106
2020-09-08[vfs] overlayfs: Fix socket tests.Ayush Ranjan
- BindSocketThenOpen test was expecting the incorrect error when opening a socket. Fixed that. - VirtualFilesystem.BindEndpointAt should not require pop.Path.Begin.Ok() because the filesystem implementations do not need to walk to the parent dentry. This check also exists for MknodAt, MkdirAt, RmdirAt, SymlinkAt and UnlinkAt but those filesystem implementations also need to walk to the parent denty. So that check is valid. Added some syscall tests to test this. PiperOrigin-RevId: 330625220
2020-09-08Honor readonly flag for root mountFabricio Voznika
Updates #1487 PiperOrigin-RevId: 330580699
2020-09-08[vfs] Capitalize x in the {Get/Set/Remove/List}xattr functions.Ayush Ranjan
PiperOrigin-RevId: 330554450
2020-09-02[vfs] Implement xattr for overlayfs.Ayush Ranjan
PiperOrigin-RevId: 329825497
2020-08-28Implement StatFS for various VFS2 filesystems.Rahat Mahmood
This mainly involved enabling kernfs' client filesystems to provide a StatFS implementation. Fixes #3411, #3515. PiperOrigin-RevId: 329009864
2020-08-25Use new reference count utility throughout gvisor.Dean Deng
This uses the refs_vfs2 template in vfs2 as well as objects common to vfs1 and vfs2. Note that vfs1-only refcounts are not replaced, since vfs1 will be deleted soon anyway. The following structs now use the new tool, with leak check enabled: devpts:rootInode fuse:inode kernfs:Dentry kernfs:dir kernfs:readonlyDir kernfs:StaticDirectory proc:fdDirInode proc:fdInfoDirInode proc:subtasksInode proc:taskInode proc:tasksInode vfs:FileDescription vfs:MountNamespace vfs:Filesystem sys:dir kernel:FSContext kernel:ProcessGroup kernel:Session shm:Shm mm:aioMappable mm:SpecialMappable transport:queue And the following use the template, but because they currently are not leak checked, a TODO is left instead of enabling leak check in this patch: kernel:FDTable tun:tunEndpoint Updates #1486. PiperOrigin-RevId: 328460377
2020-08-25Change "Fd" member to "FD" according to convensiongVisor bot
PiperOrigin-RevId: 328374775
2020-08-24Bump build constraints to 1.17Michael Pratt
This enables pre-release testing with 1.16. The intention is to replace these with a nogo check before the next release. PiperOrigin-RevId: 328193911
2020-08-24Update inotify documentation for gofer filesystem.Dean Deng
We now allow hard links to be created within gofer fs (see github.com/google/gvisor/commit/f20e63e31b56784c596897e86f03441f9d05f567). Update the inotify documentation accordingly. PiperOrigin-RevId: 328177485
2020-08-21[vfs] Allow mountpoint to be an existing non-directory.Ayush Ranjan
Unlike linux mount(2), OCI spec allows mounting on top of an existing non-directory file. PiperOrigin-RevId: 327914342
2020-08-21Provide fdReader/Writer for FileDescriptiongVisor bot
fdReader/Writer implements io.Reader/Writer so that they can be passed to Merkle tree library. PiperOrigin-RevId: 327901376
2020-08-21Internal change.gVisor bot
PiperOrigin-RevId: 327892274
2020-08-21Make mounts ReadWrite first, then later change to ReadOnly.Nicolas Lacasse
This lets us create "synthetic" mountpoint directories in ReadOnly mounts during VFS setup. Also add context.WithMountNamespace, as some filesystems (like overlay) require a MountNamespace on ctx to handle vfs.Filesystem Operations. PiperOrigin-RevId: 327874971
2020-08-20[vfs] Create recursive dir creation util.Ayush Ranjan
Refactored the recursive dir creation util in runsc/boot/vfs.go to be more flexible. PiperOrigin-RevId: 327719100
2020-08-20Consistent precondition formattingMichael Pratt
Our "Preconditions:" blocks are very useful to determine the input invariants, but they are bit inconsistent throughout the codebase, which makes them harder to read (particularly cases with 5+ conditions in a single paragraph). I've reformatted all of the cases to fit in simple rules: 1. Cases with a single condition are placed on a single line. 2. Cases with multiple conditions are placed in a bulleted list. This format has been added to the style guide. I've also mentioned "Postconditions:", though those are much less frequently used, and all uses already match this style. PiperOrigin-RevId: 327687465
2020-08-20Fix tabs in lock-ordering doc.Nicolas Lacasse
PiperOrigin-RevId: 327654207
2020-08-19Return appropriate errors when file locking is unsuccessful.Dean Deng
test_eintr now passes in the Python runtime tests. Updates #3515. PiperOrigin-RevId: 327441081
2020-08-18Don't set atime if mount is readonlyFabricio Voznika
Updates #1035 PiperOrigin-RevId: 327351475
2020-08-12Fix race in vfs.FileDescription.statusFlagFabricio Voznika
PiperOrigin-RevId: 326270643
2020-08-07Support separate read/write handles in fsimpl/gofer.dentry.Jamie Liu
PiperOrigin-RevId: 325490674
2020-08-03Plumbing context.Context to DecRef() and Release().Nayana Bidari
context is passed to DecRef() and Release() which is needed for SO_LINGER implementation. PiperOrigin-RevId: 324672584
2020-07-31Clean up vfs2 fallocate.Dean Deng
Move to setstat.go and add a FileDescription wrapper method. PiperOrigin-RevId: 324165277
2020-07-29Force registration for EPOLLHUP, not EPOLLRDHUP, in vfs2's epoll.Jamie Liu
Compare Linux's fs/eventpoll.c:do_epoll_ctl(). I don't know where EPOLLRDHUP came from. PiperOrigin-RevId: 323874419
2020-07-23Add permission checks to vfs2 truncate.Dean Deng
- Check write permission on truncate(2). Unlike ftruncate(2), truncate(2) fails if the user does not have write permissions on the file. - For gofers under InteropModeShared, check file type before making a truncate request. We should fail early and avoid making an rpc when possible. Furthermore, depending on the remote host's failure may give us unexpected behavior--if the host converts the truncate request to an ftruncate syscall on an open fd, we will get EINVAL instead of EISDIR. Updates #2923. PiperOrigin-RevId: 322913569
2020-07-23Added stub FUSE filesystemRidwan Sharif
Allow FUSE filesystems to be mounted using libfuse. The appropriate flags and mount options are parsed and understood by fusefs.
2020-07-13Clean up inotify comments.Dean Deng
PiperOrigin-RevId: 321028238
2020-07-08Add shared mount hints to VFS2Fabricio Voznika
Container restart test is disabled for VFS2 for now. Updates #1487 PiperOrigin-RevId: 320296401
2020-07-01Port fallocate to VFS2.Zach Koopmans
PiperOrigin-RevId: 319283715
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-05 08:00:46 +0000