summaryrefslogtreecommitdiffhomepage
path: root/runsc/fsgofer/fsgofer.go
AgeCommit message (Collapse)Author
2021-05-13Fix file descriptor leak in MultiGetAttrTiwei Bie
We need to make sure that all children are closed before return. But the last child saved in parent isn't closed after we successfully iterate all the files in "names". This patch fixes this issue. Fixes #5982 Signed-off-by: Tiwei Bie <tiwei.btw@antgroup.com>
2021-04-28Automated rollback of changelist 369686285Fabricio Voznika
PiperOrigin-RevId: 371015541
2021-04-21Automated rollback of changelist 369325957Michael Pratt
PiperOrigin-RevId: 369686285
2021-04-19Add MultiGetAttr message to 9PFabricio Voznika
While using remote-validation, the vast majority of time spent during FS operations is re-walking the path to check for modifications and then closing the file given that in most cases it has not been modified externally. This change introduces a new 9P message called MultiGetAttr which bulks query attributes of several files in one shot. The returned attributes are then used to update cached dentries before they are walked. File attributes are updated for files that still exist. Dentries that have been deleted are removed from the cache. And negative cache entries are removed if a new file/directory was created externally. Similarly, synthetic dentries are replaced if a file/directory is created externally. The bulk update needs to be carefull not to follow symlinks, cross mount points, because the gofer doesn't know how to resolve symlinks and where mounts points are located. It also doesn't walk to the parent ("..") to avoid deadlocks. Here are the results: Workload VFS1 VFS2 Change bazel action 115s 70s 28.8s Stat/100 11,043us 7,623us 974us Updates #1638 PiperOrigin-RevId: 369325957
2021-03-23Allow FSETXATTR/FGETXATTR host calls for VerityChong Cai
These host calls are needed for Verity fs to generate/verify hashes. PiperOrigin-RevId: 364598180
2021-03-08Internal change.Chong Cai
PiperOrigin-RevId: 361689477
2021-01-22Remove dependency to abi/linuxFabricio Voznika
abi package is to be used by the Sentry to implement the Linux ABI. Code dealing with the host should use x/sys/unix. PiperOrigin-RevId: 353272679
2021-01-21Fix ownership change logicFabricio Voznika
Previously fsgofer was skipping chown call if the uid and gid were the same as the current user/group. However, when setgid is set, the group may not be the same as the caller. Instead, compare the actual uid/gid of the file after it has been created and change ownership only if needed. Updates #180 PiperOrigin-RevId: 353118733
2021-01-12Fix simple mistakes identified by goreportcard.Adin Scannell
These are primarily simplification and lint mistakes. However, minor fixes are also included and tests added where appropriate. PiperOrigin-RevId: 351425971
2020-12-15fsgofer optimizationsFabricio Voznika
- Skip chown call in case owner change is not needed - Skip filepath.Clean() calls when joining paths - Pass unix.Stat_t by value to reduce runtime.duffcopy calls. This change allows for better inlining in localFile.walk(). Change Baseline Improvement BenchmarkWalkOne-6 2912 ns/op 3082 ns/op 5.5% BenchmarkCreate-6 15915 ns/op 19126 ns/op 16.8% BenchmarkCreateDiffOwner-6 18795 ns/op 19741 ns/op 4.8% PiperOrigin-RevId: 347667833
2020-09-17Remove option to panic goferFabricio Voznika
Gofer panics are suppressed by p9 server and an error is returned to the caller, making it effectively the same as returning EROFS. PiperOrigin-RevId: 332282959
2020-09-01Implement setattr+clunk in 9PFabricio Voznika
This is to cover the common pattern: open->read/write->close, where SetAttr needs to be called to update atime/mtime before the file is closed. Benchmark results: BM_OpenReadClose/10240 CPU setattr+clunk: 63783 ns VFS2: 68109 ns VFS1: 72507 ns Updates #1198 PiperOrigin-RevId: 329628461
2020-08-19Remove path walk from localFile.MknodFabricio Voznika
Replace mknod call with mknodat equivalent to protect against symlink attacks. Also added Mknod tests. Remove goferfs reliance on gofer to check for file existence before creating a synthetic entry. Updates #2923 PiperOrigin-RevId: 327544516
2020-08-18Return EROFS if mount is read-onlyFabricio Voznika
PiperOrigin-RevId: 327300635
2020-07-30Call lseek(0, SEEK_CUR) unconditionally in runsc fsgofer's Readdir(offset=0).Jamie Liu
9P2000.L is silent as to how readdir RPCs interact with directory mutation. The most performant option is for Treaddir with offset=0 to restart iteration, avoiding needing to walk+open+clunk a new directory fid between invocations of getdents64(2), and the VFS2 gofer client assumes this is the case. Make this actually true for the runsc fsgofer. Fixes #3344, #3345, #3355 PiperOrigin-RevId: 324090384
2020-07-25test/syscall: run each test case in a separate network namespaceAndrei Vagin
... when it is possible. The guitar gVisorKernel*Workflow-s runs test with the local execution_method. In this case, blaze runs test cases locally without sandboxes. This means that all tests run in the same network namespace. We have a few tests which use hard-coded network ports and they can fail if one of these port will be used by someone else or by another test cases. PiperOrigin-RevId: 323137254
2020-07-24Reduce walk and open cost in fsgoferFabricio Voznika
Implement WalkGetAttr() to reuse the stat that is already needed for Walk(). In addition, cache file QID, so it doesn't need to stat the file to compute it. open(2) time improved by 10%: Baseline: 6780 ns Change: 6083 ns Also fixed file type which was not being set in all places. PiperOrigin-RevId: 323102560
2020-07-23Fix fsgofer Open() when control file is using O_PATHFabricio Voznika
Open tries to reuse the control file to save syscalls and file descriptors when opening a file. However, when the control file was opened using O_PATH (e.g. no file permission to open readonly), Open() would not check for it. PiperOrigin-RevId: 322821729
2020-07-07Fix mknod and inotify syscall testAyush Ranjan
This change fixes a few things: - creating sockets using mknod(2) is supported via vfs2 - fsgofer can create regular files via mknod(2) - mode = 0 for mknod(2) will be interpreted as regular file in vfs2 as well Updates #2923 PiperOrigin-RevId: 320074267
2020-06-16Print spec as json when --debug is enabledFabricio Voznika
The previous format skipped many important structs that are pointers, especially for cgroups. Change to print as json, removing parts of the spec that are not relevant. Also removed debug message from gofer that can be very noisy when directories are large. PiperOrigin-RevId: 316713267
2020-05-28Move Cleanup to its own packageFabricio Voznika
PiperOrigin-RevId: 313663382
2020-04-07Remove TODOs for local gofer extended attributes.Dean Deng
PiperOrigin-RevId: 305344989
2020-02-07Support listxattr and removexattr syscalls.Dean Deng
Note that these are only implemented for tmpfs, and other impls will still return EOPNOTSUPP. PiperOrigin-RevId: 293899385
2020-01-16Plumb getting/setting xattrs through InodeOperations and 9p gofer interfaces.Dean Deng
There was a very bare get/setxattr in the InodeOperations interface. Add context.Context to both, size to getxattr, and flags to setxattr. Note that extended attributes are passed around as strings in this implementation, so size is automatically encoded into the value. Size is added in getxattr so that implementations can return ERANGE if a value is larger than can fit in the user-allocated buffer. This prevents us from unnecessarily passing around an arbitrarily large xattr when the user buffer is actually too small. Don't use the existing xattrwalk and xattrcreate messages and define our own, mainly for the sake of simplicity. Extended attributes will be implemented in future commits. PiperOrigin-RevId: 290121300
2020-01-09New sync package.Ian Gudger
* Rename syncutil to sync. * Add aliases to sync types. * Replace existing usage of standard library sync package. This will make it easier to swap out synchronization primitives. For example, this will allow us to use primitives from github.com/sasha-s/go-deadlock to check for lock ordering violations. Updates #1472 PiperOrigin-RevId: 289033387
2019-12-11Finish incomplete comment.Dean Deng
PiperOrigin-RevId: 285012278
2019-11-23gofer: reduce CPU usage on GC as of frequent readdirJianfeng Tan
Refer to golang mallocgc(), each time of allocating an object > 32 KB, a gc will be triggered. When we do readdir, sentry always passes 65535, which leads to a malloc of 65535 * sizeof(p9.Direnta) > 32 KB. Considering we already use slice append, let's avoid defining the capability for this slide. Command for test: Before this change: (container)$ time tree linux-5.3.1 > /dev/null real 0m54.272s user 0m2.010s sys 0m1.740s (CPU usage of Gofer: ~30 cores) (host)$ perf top -p <pid-of-gofer> 42.57% runsc [.] runtime.gcDrain 23.41% runsc [.] runtime.(*lfstack).pop 9.74% runsc [.] runtime.greyobject 8.06% runsc [.] runtime.(*lfstack).push 4.33% runsc [.] runtime.scanobject 1.69% runsc [.] runtime.findObject 1.12% runsc [.] runtime.findrunnable 0.69% runsc [.] runtime.runqgrab ... (host)$ mkdir test && cd test (host)$ for i in `seq 1 65536`; do mkdir $i; done (container)$ time ls test/ > /dev/null real 2m10.934s user 0m0.280s sys 0m4.260s (CPU usage of Gofer: ~1 core) After this change: (container)$ time tree linux-5.3.1 > /dev/null real 0m22.465s user 0m1.270s sys 0m1.310s (CPU usage of Gofer: ~1 core) $ perf top -p <pid-of-gofer> 20.57% runsc [.] runtime.gcDrain 7.15% runsc [.] runtime.(*lfstack).pop 4.11% runsc [.] runtime.scanobject 3.78% runsc [.] runtime.greyobject 2.78% runsc [.] runtime.(*lfstack).push ... (host)$ mkdir test && cd test (host)$ for i in `seq 1 65536`; do mkdir $i; done (container)$ time ls test/ > /dev/null real 0m13.338s user 0m0.190s sys 0m3.980s (CPU usage of Gofer: ~0.8 core) Fixes #898 Signed-off-by: Jianfeng Tan <henry.tjf@antfin.com>
2019-11-06Add p9.OpenTruncate.Jamie Liu
This is required to implement O_TRUNC correctly on filesystems backed by gofers. 9P2000.L: "lopen prepares fid for file I/O. flags contains Linux open(2) flags bits, e.g. O_RDONLY, O_RDWR, O_WRONLY." open(2): "The argument flags must include one of the following access modes: O_RDONLY, O_WRONLY, or O_RDWR. ... In addition, zero or more file creation flags and file status flags can be bitwise-or'd in flags." The reference 9P2000.L implementation also appears to expect arbitrary flags, not just access modes, in Tlopen.flags: https://github.com/chaos/diod/blob/master/diod/ops.c#L703 PiperOrigin-RevId: 278972683
2019-10-28Cast the Stat_t.Nlink to uint64 on arm64.Haibo Xu
Since the syscall.Stat_t.Nlink is defined as different types on amd64 and arm64(uint64 and uint32 respectively), we need to cast them to a unified uint64 type in gVisor code. Signed-off-by: Haibo Xu <haibo.xu@arm.com> Change-Id: I7542b99b195c708f3fc49b1cbe6adebdd2f6e96b
2019-10-18Cleanup host UDS supportMichael Pratt
This change fixes several issues with the fsgofer host UDS support. Notably, it adds support for SOCK_SEQPACKET and SOCK_DGRAM sockets [1]. It also fixes unsafe use of unet.Socket, which could cause a panic if Socket.FD is called when err != nil, and calls to Socket.FD with nothing to prevent the garbage collector from destroying and closing the socket. A set of tests is added to exercise host UDS access. This required extracting most of the syscall test runner into a library that can be used by custom tests. Updates #235 Updates #1003 [1] N.B. SOCK_DGRAM sockets are likely not particularly useful, as a server can only reply to a client that binds first. We don't allow bind, so these are unlikely to be used. PiperOrigin-RevId: 275558502
2019-10-15Make Attach no longer a special snowflakeMichael Pratt
fsgofer.attachPoint.Attach has a bunch of funky special logic to create a RW file or connect a socket rather than creating a standard control file like localFile.Walk. This is unecessary and error-prone, as the attach point still has to go through Open or Connect which will properly convert the control file to something usable. As such, switch the logic to be equivalent to a simple Walk. Updates #235 PiperOrigin-RevId: 274827872
2019-09-26Disallow opening of sockets if --fsgofer-host-uds=falseFabricio Voznika
Updates #235 PiperOrigin-RevId: 271475319
2019-09-25Merge pull request #765 from trailofbits:uds_supportgVisor bot
PiperOrigin-RevId: 271235134
2019-09-24Refactor command line options and remove the allowed terminology for udsRobert Tonic
2019-09-19Fix documentation, clean up seccomp filter installation, rename helpers.Robert Tonic
Filter installation has been streamlined and functions renamed. Documentation has been fixed to be standards compliant, and missing documentation added. gofmt has also been applied to modified files.
2019-09-19Place the host UDS mounting behind --fsgofer-host-uds-allowed.Robert Tonic
This commit allows the use of the `--fsgofer-host-uds-allowed` flag to enable mounting sockets and add the appropriate seccomp filters.
2019-09-05Apply go fmt to the fsgofer changes.Robert Tonic
2019-09-05Remove seccomp permissions, and clean up the Attach logic.Robert Tonic
2019-08-27First pass at implementing Unix Domain Socket support. No tests.Robert Tonic
This commit adds support for detecting the socket file type, connecting to a Unix Domain Socket, and providing bidirectional communication (without file descriptor transfer support).
2019-08-13Fix file mode check in fsgofer AttachFabricio Voznika
PiperOrigin-RevId: 263189654
2019-06-27Fix various spelling issues in the documentationMichael Pratt
Addresses obvious typos, in the documentation only. COPYBARA_INTEGRATE_REVIEW=https://github.com/google/gvisor/pull/443 from Pixep:fix/documentation-spelling 4d0688164eafaf0b3010e5f4824b35d1e7176d65 PiperOrigin-RevId: 255477779
2019-06-24fsgopher: reopen files via /proc/self/fdAndrei Vagin
When we reopen file by path, we can't be sure that we will open exactly the same file. The file can be deleted and another one with the same name can be created. PiperOrigin-RevId: 254898594
2019-06-13Update canonical repository.Adin Scannell
This can be merged after: https://github.com/google/gvisor-website/pull/77 or https://github.com/google/gvisor-website/pull/78 PiperOrigin-RevId: 253132620
2019-05-17Return EPERM for mknodMichael Pratt
This more directly matches what Linux does with unsupported nodes. PiperOrigin-RevId: 248780425 Change-Id: I17f3dd0b244f6dc4eb00e2e42344851b8367fbec
2019-05-09Implement fallocate(2)Fabricio Voznika
Closes #225 PiperOrigin-RevId: 247508791 Change-Id: I04f47cf2770b30043e5a272aba4ba6e11d0476cc
2019-05-03gofer: don't leak file descriptorsAndrei Vagin
Fixes #219 PiperOrigin-RevId: 246568639 Change-Id: Ic7afd15dde922638d77f6429c508d1cbe2e4288a
2019-04-29Change copyright notice to "The gVisor Authors"Michael Pratt
Based on the guidelines at https://opensource.google.com/docs/releasing/authors/. 1. $ rg -l "Google LLC" | xargs sed -i 's/Google LLC.*/The gVisor Authors./' 2. Manual fixup of "Google Inc" references. 3. Add AUTHORS file. Authors may request to be added to this file. 4. Point netstack AUTHORS to gVisor AUTHORS. Drop CONTRIBUTORS. Fixes #209 PiperOrigin-RevId: 245823212 Change-Id: I64530b24ad021a7d683137459cafc510f5ee1de9
2019-04-23Remember file position during Readdir()Fabricio Voznika
The caller must call Readdir() at least twice to detect EOF. The old code was always restarting the directory search and then skipping elements already seen, effectively doubling the cost to read a directory. The code now remembers the last offset and doesn't reposition the cursor if next request comes at the same offset. PiperOrigin-RevId: 244957816 Change-Id: If21a8dc68b76614adbcf4301439adfda40f2643f
2019-04-23Replace os.File with fd.FD in fsgoferFabricio Voznika
os.NewFile() accounts for 38% of CPU time in localFile.Walk(). This change switchs to use fd.FD which is much cheaper to create. Now, fd.New() in localFile.Walk() accounts for only 4%. PiperOrigin-RevId: 244944983 Change-Id: Ic892df96cf2633e78ad379227a213cb93ee0ca46
2019-04-01Automated rollback of changelist 240657604Fabricio Voznika
PiperOrigin-RevId: 241434161 Change-Id: I9ec734e50cef5b39203e8bf37de2d91d24943f1e
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-04 21:10:28 +0000