summaryrefslogtreecommitdiffhomepage
path: root/tools
AgeCommit message (Collapse)Author
2021-04-14[syserror] Remove syserror from go_marshalZach Koopmans
PiperOrigin-RevId: 368470656
2021-04-14Automatically enforce limited netstack dependenciesKevin Krakauer
Netstack is supposed to be somewhat independent of the rest of gVisor, and others should be able to use it without pulling in excessive dependencies. Currently, there is no way to fight dependency creep besides careful code review. This change introduces a test rule `netstack_deps_check` that ensures the target only relies on gVisor targets and a short allowlist of external dependencies. Users who add a dependency will see an error and have to manually update the allowlist. The set of packages to test comes from //runsc, as it uses packages we would expect users to commonly rely on. It was generated via: $ find ./runsc -name BUILD | xargs grep tcpip | awk '{print $2}' | sort | uniq (Note: We considered giving //pkg/tcpip it's own go.mod, but this breaks go tooling.) PiperOrigin-RevId: 368456711
2021-04-08Drop unused escapes information.Adin Scannell
PiperOrigin-RevId: 367517305
2021-04-02Internal change.gVisor bot
PiperOrigin-RevId: 366555466
2021-03-29[syserror] Split usermem packageZach Koopmans
Split usermem package to help remove syserror dependency in go_marshal. New hostarch package contains code not dependent on syserror. PiperOrigin-RevId: 365651233
2021-03-23Skip checklocks analysis for stateify generated code.Rahat Mahmood
Stateify methods are always called without holding the appropriate locks. The system is paused and we know there will be no mutations when we call Save/Load, so this is perfectly safe. However, checklocks can't know about this, and it will always complain. Mark stateify generated methods that touch struct fields as "checklocksignore" to avoid this. PiperOrigin-RevId: 364610241
2021-03-22[lisa] Support dynamic types for all types.Ayush Ranjan
We were only supporting dynamic struct types. With this change, users can make any type dynamic. The tool (correctly) blindly just generates the remaining methods needed to implement Marshallable using the 3 methods defined by the user on the dynamic type. This is helpful in situations like: type StringArray []string Added a test for such a use case. PiperOrigin-RevId: 364463164
2021-03-22Emit comment about build tags in gomarshal generated files.Rahat Mahmood
This may be useful for tracking down where build tags come from and understanding tag import issues in generated files. PiperOrigin-RevId: 364374931
2021-03-12Merge pull request #5663 from avagin:apt-repogVisor bot
PiperOrigin-RevId: 362545342
2021-03-11make/release: Sign a package only if it isn't signed yet.Andrei Vagin
We can generate more than one apt repo for the same package. If we will sign a package again, its file will be changed and all hashes that have been generated before will be invalid.
2021-03-11Implement Merkle tree generate tool binaryChong Cai
This binary is used to recursively enable and generate Merkle tree files for all files and directories in a file system from inside a gVisor sandbox. PiperOrigin-RevId: 362418770
2021-03-08[lisa] Do not generate any tests for dynamic types.Ayush Ranjan
The dynamic type user defines the marshalling logic, so we don't need to test for things like alignment, absence of slices, etc. For dynamic types, the go_marshal generator just generates the missing methods required to implement marshal.Marshallable. PiperOrigin-RevId: 361676311
2021-03-03Add checklocks analyzer.Bhasker Hariharan
This validates that struct fields if annotated with "// checklocks:mu" where "mu" is a mutex field in the same struct then access to the field is only done with "mu" locked. All types that are guarded by a mutex must be annotated with // +checklocks:<mutex field name> For more details please refer to README.md. PiperOrigin-RevId: 360729328
2021-02-25Strip all non-numeric characters from version.Adin Scannell
This will fix debian packaging. Updates #5510 PiperOrigin-RevId: 359563378
2021-02-08[go-marshal] Add dynamic tag in go_marshal.Ayush Ranjan
This makes it easier to implement dynamically sized types in go-marshal. You really only need to implement MarshalBytes, UnmarshalBytes and SizeBytes to implement the entire interface. By using the `dynamic` tag, the autogenerator will generate the rest of the methods for us. This change also simplifies how KernelIPTGetEntries implements Marshallable using the newly added utility. PiperOrigin-RevId: 356397114
2021-02-03Remove gazelle noise patchTamir Duberstein
Confirmed gazelle generates no significant noise. Fix documented gazelle invocation while I'm here. PiperOrigin-RevId: 355452758
2021-02-02Update go-tools to 2020.2.1Tamir Duberstein
...and a bunch of other things as I worked through the rot. Notably: - Upgrade to bazel 4.0.0 - Upgrade to Go 1.15.7 Remove go_branch stderr suppression; this made it quite difficult to see what was failing while developing this patch. PiperOrigin-RevId: 355257833
2021-02-02Remove go_tool_library references.Adin Scannell
This is required only for the built-in bazel nogo functionality. Since we roll these targets manually via the wrappers, we don't need to use go_tool_library. The inconsistent use of these targets leads to conflicting instantiations of go_default_library and go_tool_library, which both contain the same output files. PiperOrigin-RevId: 355184975
2021-02-01Fix empty Packages file for APT repository.Adin Scannell
This change also adds an extra sanity check to the make_apt.sh script, in order to ensure that this simple mistake does not occur again. PiperOrigin-RevId: 355101754
2021-02-01Remove Go cache on failure.Adin Scannell
It's unclear why permissions wind up corrupted, but these can be cleared on any failure, similar to the bazel cache itself: https://buildkite.com/gvisor/pipeline/builds/2304#_ PiperOrigin-RevId: 355057421
2021-01-22[infra] Fix Build Github action failure.Ayush Ranjan
The GH Build action has been failing with the error message: ``` --- BUILD -c opt //runsc tee: /proc/self/fd/2: No such device or address ``` tee /dev/fd/2 seems to be the canonical way of copying stdin to stderr. So use that instead. PiperOrigin-RevId: 353259087
2021-01-12Fix Go branch building.Adin Scannell
Files removed from the working tree were not being properly removed from the branch, leading to symbol conflicts while building. This requires the change to 'git add --all' in the tools/go_branch.sh script. But why was this not caught by CI? The "git clean -f" command by default only cleans files in the current working directory. In order to clean the whole tree recursively, we need to specify a pathspec, which is ".". In addition to these fixes, re-add the "go tests" command to help prevent this from happening again, since merges on the Go branch will happen in GitHub actions for simplicity. The Go test is retained in BuildKite. PiperOrigin-RevId: 351503804
2021-01-12Delete shim v1Fabricio Voznika
gvisor-containerd-shim is not compatible with containerd 1.1 or earlier. Starting from containerd 1.2, shim v2 is the preferred interface. PiperOrigin-RevId: 351485556
2021-01-12Drop shutdown in docker run.Adin Scannell
This can race and cause issues. Instead, a manual shutdown can be done via 'bazel-shutdown' if required for specific cases. The ARM64 builds are now done using cross-compilation, so this hack is not necessary. PiperOrigin-RevId: 351477908
2021-01-12Fix simple mistakes identified by goreportcard.Adin Scannell
These are primarily simplification and lint mistakes. However, minor fixes are also included and tests added where appropriate. PiperOrigin-RevId: 351425971
2021-01-11Fix Go branch for arm64.Adin Scannell
This requires several changes: * Templates must preserve relevant tags. * Pagetables templates are split into two targets, each preserving tags. * The binary VDSO is similarly split into two targets, with some juggling. * The top level tools/go_branch.sh now does a crossbuild of ARM64 as well, and checks and merges the results of the two branches together. Fixes #5178 PiperOrigin-RevId: 351304330
2021-01-08Support releasing aarch64 builds.Adin Scannell
This change works around an issue in rules_pkg, described here: https://github.com/bazelbuild/rules_pkg/pull/263 PiperOrigin-RevId: 350869030
2021-01-07Add ARM smoke testAndrei Vagin
make BAZEL_CONFIG=aarch64 arm-qemu-smoke-test Signed-off-by: Andrei Vagin <avagin@gmail.com>
2021-01-06Don't canonicalize cache directory.Adin Scannell
... Otherwise it will be mounted in via some other path, and will not be available inside the container at all. PiperOrigin-RevId: 350440843
2021-01-06Include objdump failures in test output.Michael Pratt
We log a warning if objdump fails, but this appears in the build log, not test log, which can make it hard to notice. Include it with the actual escape output as context on "(possible)" to make it more clear when something is wrong. PiperOrigin-RevId: 350355759
2021-01-05Make build command more robust.Adin Scannell
This returns all targets, and handles no targets. PiperOrigin-RevId: 350263578
2021-01-05Merge pull request #5169 from laijs:fix-typo-whicygVisor bot
PiperOrigin-RevId: 350200437
2021-01-05Add YAML validation for configuration files.Adin Scannell
For validation, the "on" key in existing YAML files is changed to a literal string. In the YAML spec, on is a keyword which encodes a boolean value, so without relying on a specific implementation the YAML files are technically not encoding an object that complies with the specification. PiperOrigin-RevId: 350172147
2020-12-30Remove remote execution support.Adin Scannell
PiperOrigin-RevId: 349616845
2020-12-30Add BuildKite annotations for failures and profiles.Adin Scannell
This change cleans up some minor Makefile issues, and adds support for BuildKite annotations on failure and on profiles being generated. These annotations will make failures very clear and link to the artifacts. This change is a stepping stone for aggregating coverage data from all individual test jobs, as this will also happen in .buildkite/annotate.sh. PiperOrigin-RevId: 349606598
2020-12-30Use a stable ordering for generated types.Adin Scannell
Otherwise this pollutes the 'go' branch and doesn't conform to standards for generate bazel files. PiperOrigin-RevId: 349605037
2020-12-29Exclude changing generated file from go_branch.Adin Scannell
PiperOrigin-RevId: 349491589
2020-12-27typo: change whicy to whichLai Jiangshan
Signed-off-by: Lai Jiangshan <jiangshan.ljs@antfin.com>
2020-12-11Remove existing nogo exceptions.Adin Scannell
PiperOrigin-RevId: 347047550
2020-12-11Fix parser to include iterations.Zach Koopmans
PiperOrigin-RevId: 347038652
2020-12-10Fix typo in go template error messages.Dean Deng
PiperOrigin-RevId: 346878344
2020-12-09Tweak aarch64 support.Adin Scannell
A few images were broken with respect to aarch64. We should now be able to run push-all-images with ARCH=aarch64 as part of the regular continuous integration builds, and add aarch64 smoke tests (via user emulation for now) to the regular test suite (future). PiperOrigin-RevId: 346685462
2020-12-09Add //pkg/sync:generic_atomicptrmap.Jamie Liu
AtomicPtrMap is a generic concurrent map from arbitrary keys to arbitrary pointer values. Benchmarks: name time/op StoreDelete/RWMutexMap-12 335ns ± 1% StoreDelete/SyncMap-12 705ns ± 3% StoreDelete/AtomicPtrMap-12 287ns ± 4% StoreDelete/AtomicPtrMapSharded-12 289ns ± 1% LoadOrStoreDelete/RWMutexMap-12 342ns ± 2% LoadOrStoreDelete/SyncMap-12 662ns ± 2% LoadOrStoreDelete/AtomicPtrMap-12 290ns ± 7% LoadOrStoreDelete/AtomicPtrMapSharded-12 293ns ± 2% LookupPositive/RWMutexMap-12 101ns ±26% LookupPositive/SyncMap-12 202ns ± 2% LookupPositive/AtomicPtrMap-12 71.1ns ± 2% LookupPositive/AtomicPtrMapSharded-12 73.2ns ± 1% LookupNegative/RWMutexMap-12 119ns ± 1% LookupNegative/SyncMap-12 154ns ± 1% LookupNegative/AtomicPtrMap-12 84.7ns ± 3% LookupNegative/AtomicPtrMapSharded-12 86.8ns ± 1% Concurrent/FixedKeys_1PercentWrites_RWMutexMap-12 1.32µs ± 2% Concurrent/FixedKeys_1PercentWrites_SyncMap-12 52.7ns ±10% Concurrent/FixedKeys_1PercentWrites_AtomicPtrMap-12 31.8ns ±20% Concurrent/FixedKeys_1PercentWrites_AtomicPtrMapSharded-12 24.0ns ±15% Concurrent/FixedKeys_10PercentWrites_RWMutexMap-12 860ns ± 3% Concurrent/FixedKeys_10PercentWrites_SyncMap-12 68.8ns ±20% Concurrent/FixedKeys_10PercentWrites_AtomicPtrMap-12 98.6ns ± 7% Concurrent/FixedKeys_10PercentWrites_AtomicPtrMapSharded-12 42.0ns ±25% Concurrent/FixedKeys_50PercentWrites_RWMutexMap-12 1.17µs ± 3% Concurrent/FixedKeys_50PercentWrites_SyncMap-12 136ns ±34% Concurrent/FixedKeys_50PercentWrites_AtomicPtrMap-12 286ns ± 3% Concurrent/FixedKeys_50PercentWrites_AtomicPtrMapSharded-12 115ns ±35% Concurrent/ChangingKeys_1PercentWrites_RWMutexMap-12 1.27µs ± 2% Concurrent/ChangingKeys_1PercentWrites_SyncMap-12 5.01µs ± 3% Concurrent/ChangingKeys_1PercentWrites_AtomicPtrMap-12 38.1ns ± 3% Concurrent/ChangingKeys_1PercentWrites_AtomicPtrMapSharded-12 22.6ns ± 2% Concurrent/ChangingKeys_10PercentWrites_RWMutexMap-12 1.08µs ± 2% Concurrent/ChangingKeys_10PercentWrites_SyncMap-12 5.97µs ± 1% Concurrent/ChangingKeys_10PercentWrites_AtomicPtrMap-12 390ns ± 2% Concurrent/ChangingKeys_10PercentWrites_AtomicPtrMapSharded-12 93.6ns ± 1% Concurrent/ChangingKeys_50PercentWrites_RWMutexMap-12 1.77µs ± 2% Concurrent/ChangingKeys_50PercentWrites_SyncMap-12 8.07µs ± 2% Concurrent/ChangingKeys_50PercentWrites_AtomicPtrMap-12 1.61µs ± 2% Concurrent/ChangingKeys_50PercentWrites_AtomicPtrMapSharded-12 386ns ± 1% Updates #231 PiperOrigin-RevId: 346614776
2020-12-09Refactor the Makefile to avoid recursive Make.Adin Scannell
Recursive make is difficult to follow and debug. Drop this by using internal functions, which, while difficult, are easier than trying to following recursive invokations. Further simplify the Makefile by collapsing the image bits and removing the tools/vm directory, which is effectively unused. Fixes #4952 PiperOrigin-RevId: 346569133
2020-12-09Prepare for supporting cross compilation.Andrei Vagin
PiperOrigin-RevId: 346496532
2020-12-03Support partitions for other tests.Adin Scannell
PiperOrigin-RevId: 345399936
2020-12-02Fix containerd.sh for later Ubuntu and Debian-based distributions.Adin Scannell
PiperOrigin-RevId: 345245285
2020-11-30Ensure containerd is used from installed location.Adin Scannell
Currently, if containerd is installed locally via tools/installers/containerd, then it will not necessarily be used if containerd is installed in the system path. This means that the existing containerd tests are all likely broken. Also, use libbtrfs-dev instead of btrfs-tools, which is not actually required. PiperOrigin-RevId: 344879109
2020-11-23Fix bad Makefile variable reference.Adin Scannell
PiperOrigin-RevId: 343946859
2020-11-23Clean up build output.Adin Scannell
This change also simplifies and documents the build_cmd pipeline, and reduces general noise for debugging Makefile issues. It also drops the mapping for /etc/docker/daemon.json, which if it does not exist initially will create this as a directory (causing lots of confusion and breaks). PiperOrigin-RevId: 343932456
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-08 18:54:48 +0000