summaryrefslogtreecommitdiffhomepage
path: root/test/runtimes
AgeCommit message (Collapse)Author
2020-09-29Stop depending on go_binary targets.Adin Scannell
Closes #3374 PiperOrigin-RevId: 334505627
2020-09-25Disable flaky java11 tests.Jamie Liu
Regarding ThreadCpuTimeArray.java: The test starts 10 threads, each of which does some computation, then blocks. When all threads are blocked, the test sleeps for 200ms, then checks that less than 100ns of CPU time in userspace elapse over the course of the sleep; AFAICT, the 100ns of slop is because a thread indicates that it's in the WAITING state before it actually blocks, and because signals can cause threads to be temporarily woken. gVisor's CPU clocks have a granularity of 10ms (the interval of Kernel.cpuClockTicker is //pkg/abi/linux.ClockTick), so a single tick pushes the test over the threshold. PiperOrigin-RevId: 333830287
2020-09-18Disable PHP test ext/standard/tests/streams/proc_open_bug60120.phpt.Jamie Liu
PiperOrigin-RevId: 332521647
2020-09-17Disable nodejs12.4 test async-hooks/test-statwatcher.Jamie Liu
PiperOrigin-RevId: 332281912
2020-09-16[runtime tests] Add documentation.Ayush Ranjan
Added a README describing what these tests are, how they work and how to run them locally. Also reorganized the exclude files into a directory. PiperOrigin-RevId: 332079697
2020-09-02[runtime tests] Exclude flaky nodejs testAyush Ranjan
PiperOrigin-RevId: 329749191
2020-08-26[runtime-tests] Exclude flaky nodejs test.Ayush Ranjan
PiperOrigin-RevId: 328579755
2020-08-25Link to PHP bug for disabled disk space tests.Jamie Liu
PiperOrigin-RevId: 328410399
2020-08-25Add nogo support to go_binary and go_test targets.Adin Scannell
Updates #3374 PiperOrigin-RevId: 328378700
2020-08-25Disable PHP disk space tests.Jamie Liu
These tests print disk_free_space()/disk_total_space() and expect the printed result to be an integer (despite the fact that both the documented and returned type is float). After cl/297213789, free/total disk space on tmpfs is sufficiently large that PHP prints the result in scientific notation instead: ========DIFF======== 012+ float(9.2233720368548E+18) 013+ float(9.2233720368548E+18) 012- float(%d) 013- float(%d) ========DONE======== FAIL disk_total_space() and disk_free_space() tests [ext/standard/tests/file/disk.phpt] PiperOrigin-RevId: 328349906
2020-08-07[runtime tests] Exclude flaky/failing testsAyush Ranjan
PiperOrigin-RevId: 325481011
2020-08-06Add bzl_library rules for .bzl files without one.Adin Scannell
PiperOrigin-RevId: 325280924
2020-08-06[runtime tests] Exclude failing python and java testsAyush Ranjan
test_httplib was un-excluded in cl/325137657. Exclude it again. test_httplib fails consistently when running in a kokoro job. Could not repro with Docker. There are some java tests that pass locally on Docker but fail when running within a kokoro job. Also make shard count a multiple of 8 to minimize idle cores and minimize test time. PiperOrigin-RevId: 325267071
2020-08-05[runtime tests] Update python tests.Ayush Ranjan
- un-exclude passing tests to increase testing surface. - create/update bugs for tests that fail on runsc but pass on runc. PiperOrigin-RevId: 325137657
2020-08-05[runtime tests] Update bugs and un-exclude passing tests.Ayush Ranjan
PiperOrigin-RevId: 325045486
2020-08-04[runtime tests] Exclude flaky tests.Ayush Ranjan
PiperOrigin-RevId: 324923599
2020-08-04[runtime tests] Fix nodejs runtime tests.Ayush Ranjan
- Exclude flaky tests. - Bump timeout. - Un-exclude passing tests to increase testing surface. - Create/Update bugs for tests that pass on runc but fail on runsc. PiperOrigin-RevId: 324830840
2020-08-03Exclude flaky nodejs runtime test.Dean Deng
PiperOrigin-RevId: 324738840
2020-07-31[runtime tests] Enhance java runtime test.Ayush Ranjan
- Added a bunch of helpful options which help in speeding up the test and providing useful output. - Unexcluded passing tests and updated bugs. Excluded tests which were failing. - Increased the batch size for java tests so that we can take advantage of the shared JVMs. The running time of the tests decreased from 3+ hours (I don't know the exact running time because this test has always timed out after 3 hours) to 1 hour 15 minutes. We can reliably run this a CI kokoro job. PiperOrigin-RevId: 324301503
2020-07-30[runtime tests] go language test enhancementAyush Ranjan
- Unexported some passing tests. This will increase the testing surface and will be especially helpful when this is enabled for vfs2. - Run tool tests with -v (verbose output). We only print the output when a test fails so this should not clutter the output. - Run tool tests with "-no-rebuild" flag. - Surround test name with appropriate regex, i.e. ^testname$. This will ensure that only that test is run. Earlier running go_test:os would also run go_test:os/exec, go_test:os/signal, go_test:os/user. This should help speed up the tests as we do not run the same test multiple times anymore. - Updated bugs. Updates #3191 PiperOrigin-RevId: 324028878
2020-07-28[Runtime Tests] Exclude flaky/failing testsAyush Ranjan
PiperOrigin-RevId: 323632510
2020-07-17Runtime tests are enormousAyush Ranjan
PiperOrigin-RevId: 321885126
2020-07-17Automated rollback of changelist 321647645Ayush Ranjan
PiperOrigin-RevId: 321808673
2020-07-16Port runtime tests to use go_testAyush Ranjan
PiperOrigin-RevId: 321647645
2020-07-11Runtime test batch executorAyush Ranjan
Earlier we were docker exec-ing each test at a time. However invoking the test framework has a fixed overhead which made it infeasible to make the runtime tests run as presubmits. This change now executes tests in batches of 50 (can be altered). This really speeds up testing process. With this change, the following tests can be run in reasonable times: - Go - Nodejs - Php - Python PiperOrigin-RevId: 320763916
2020-07-08Move all tests to new docker API.Zach Koopmans
Moves following to new dockerutil API: - //test/e2e:integration_test - //test/image:image_test - //test/iptables:iptables_test - //test/root:root_test - //test/packetimpact:packetimpact_test PiperOrigin-RevId: 320253118
2020-07-08Enable shards in runtime test runner.Ayush Ranjan
Fixed an issue with the runtime test runner which enables us to run tests in shards. We had to touch the status file as indicated by an env var. PiperOrigin-RevId: 320236205
2020-06-17Move mount configutation to RunOptsFabricio Voznika
Separate mount configuration from links and move it to RunOpts, like the other options. PiperOrigin-RevId: 317010158
2020-06-15Remove blacklist from //test/runtimesMichael Pratt
Updates #2972 PiperOrigin-RevId: 316534165
2020-06-08test/runtimes/proctor: remove an unknown nocgo attribute from go_test ruleAndrei Vagin
PiperOrigin-RevId: 315353408
2020-05-05Internal change.gVisor bot
PiperOrigin-RevId: 310001058
2020-04-24Standardize all Docker images.Adin Scannell
This change moves all Docker images to a standard location, and abstracts the build process so that they can be maintained in an automated fashion. This also allows the images to be architecture-independent. All images will now be referred to by the test framework via the canonical `gvisor.dev/images/<name>`, where `<name>` is a function of the path within the source tree. In a subsequent change, continuous integration will be added so that the images will always be correct and available locally. In the end, using `bazel` for Docker containers is simply not possible. Given that we already have the need to use `make` with the base container (for Docker), we extend this approach to get more flexibility. This change also adds a self-documenting and powerful Makefile that is intended to replace the collection of scripts in scripts. Canonical (self-documenting) targets can be added here for targets that understand which images need to be loaded and/or built. PiperOrigin-RevId: 308322438
2020-04-23Simplify Docker test infrastructure.Adin Scannell
This change adds a layer of abstraction around the internal Docker APIs, and eliminates all direct dependencies on Dockerfiles in the infrastructure. A subsequent change will automated the generation of local images (with efficient caching). Note that this change drops the use of bazel container rules, as that experiment does not seem to be viable. PiperOrigin-RevId: 308095430
2020-04-08Fix all printf formatting errors.Adin Scannell
Updates #2243
2020-02-11Fix up test/runtimes/README.md.gVisor bot
In particular, explain how to push updates to the images. PiperOrigin-RevId: 294508879
2020-02-03Reduce run time for //test/syscalls:socket_inet_loopback_test_runsc_ptrace.Ting-Yu Wang
* Tests are picked for a shard differently. It now picks one test from each block, instead of picking the whole block. This makes the same kind of tests spreads across different shards. * Reduce the number of connect() calls in TCPListenClose. PiperOrigin-RevId: 293019281
2020-01-27Standardize on tools directory.Adin Scannell
PiperOrigin-RevId: 291745021
2019-11-14Allow all runtime tests for a language to be run via a single command.Kevin Krakauer
This was intended behavior per the README, but running tests without the --test flag caused an error. Users can now omit the --test flag to run every test for a runtime. PiperOrigin-RevId: 280522025
2019-11-01Add "manual" tag back to runtime tests.Nicolas Lacasse
PiperOrigin-RevId: 277971910
2019-10-30Add Kokoro config for new runtime testsBrad Burlage
PiperOrigin-RevId: 277607217
2019-10-09Add blacklists for remaining runtime tests, and test that they parse correctly.Nicolas Lacasse
PiperOrigin-RevId: 273781112
2019-10-01Add blacklist support to the runtime test runner.Nicolas Lacasse
Tests in the blacklist will be explicitly skipped (with associated log line). Checks in a blacklist for the nodejs tests. PiperOrigin-RevId: 272272749
2019-09-23Run all runtime tests in a single container.Nicolas Lacasse
This makes them run much faster. Also cleaned up the log reporting. PiperOrigin-RevId: 270799808
2019-09-18Shard the runtime tests.Nicolas Lacasse
Default of 20 shards was arbitrary and will need fine-tuning in later CLs. PiperOrigin-RevId: 269922871
2019-09-17Follow-up fixes for image tests.Nicolas Lacasse
- Fix ARG syntax in Dockerfiles. - Fix curl commands in Dockerfiles. - Fix some paths in proctor binaries. - Check error from Walk in search helper. PiperOrigin-RevId: 269641686
2019-09-16Refactor and clean up image tests.Nicolas Lacasse
* Use multi-stage builds in Dockerfiles. * Combine all proctor binaries into a single binary. * Change the TestRunner interface to reduce code duplication. PiperOrigin-RevId: 269462101
2019-09-03Impose order on test scripts.Adin Scannell
The simple test script has gotten out of control. Shard this script into different pieces and attempt to impose order on overall test structure. This change helps lay some of the foundations for future improvements. * The runsc/test directories are moved into just test/. * The runsc/test/testutil package is split into logical pieces. * The scripts/ directory contains new top-level targets. * Each test is now responsible for building targets it requires. * The install functionality is moved into `runsc` itself for simplicity. * The existing kokoro run_tests.sh file now just calls all (can be split). After this change is merged, I will create multiple distinct workflows for Kokoro, one for each of the scripts currently targeted by `run_tests.sh` today, which should dramatically reduce the time-to-run for the Kokoro tests, and provides a better foundation for further improvements to the infrastructure. PiperOrigin-RevId: 267081397
2019-08-29Compile procter binaries during image creation.Nicolas Lacasse
Using "go run ..." in the ENTRYPOINT causes the go compiler to run each time the container is started. We can just compile the binary once as part of the image. PiperOrigin-RevId: 266212462
2019-08-20Don't run runtime tests in parallel.Nicolas Lacasse
We need real sharding, and will let Bazel handle the parallelization. That is coming soon. Until then, remove this call to t.Parallel() so that we can run the tests without eating all CPU. PiperOrigin-RevId: 264498919
2019-08-09Create tests for common.Search().Brett Landau
Using the path_test.go file built by the Golang devs as a base, tests have been created to verify the functionality of common.Search(). A mock file system is created and fake test files are generated to see if they get picked up by common.Search(). Also included in this CL is a bug fix for proctor-nodejs that was discovered using this test. proctor-nodejs used to allow multiple "-" in its test name filter. The regex has been updated to prevent this. PiperOrigin-RevId: 262647263
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-18 02:27:29 +0000