summaryrefslogtreecommitdiffhomepage
path: root/test/runtimes
AgeCommit message (Collapse)Author
2020-04-24Standardize all Docker images.Adin Scannell
This change moves all Docker images to a standard location, and abstracts the build process so that they can be maintained in an automated fashion. This also allows the images to be architecture-independent. All images will now be referred to by the test framework via the canonical `gvisor.dev/images/<name>`, where `<name>` is a function of the path within the source tree. In a subsequent change, continuous integration will be added so that the images will always be correct and available locally. In the end, using `bazel` for Docker containers is simply not possible. Given that we already have the need to use `make` with the base container (for Docker), we extend this approach to get more flexibility. This change also adds a self-documenting and powerful Makefile that is intended to replace the collection of scripts in scripts. Canonical (self-documenting) targets can be added here for targets that understand which images need to be loaded and/or built. PiperOrigin-RevId: 308322438
2020-04-23Simplify Docker test infrastructure.Adin Scannell
This change adds a layer of abstraction around the internal Docker APIs, and eliminates all direct dependencies on Dockerfiles in the infrastructure. A subsequent change will automated the generation of local images (with efficient caching). Note that this change drops the use of bazel container rules, as that experiment does not seem to be viable. PiperOrigin-RevId: 308095430
2020-04-08Fix all printf formatting errors.Adin Scannell
Updates #2243
2020-02-11Fix up test/runtimes/README.md.gVisor bot
In particular, explain how to push updates to the images. PiperOrigin-RevId: 294508879
2020-02-03Reduce run time for //test/syscalls:socket_inet_loopback_test_runsc_ptrace.Ting-Yu Wang
* Tests are picked for a shard differently. It now picks one test from each block, instead of picking the whole block. This makes the same kind of tests spreads across different shards. * Reduce the number of connect() calls in TCPListenClose. PiperOrigin-RevId: 293019281
2020-01-27Standardize on tools directory.Adin Scannell
PiperOrigin-RevId: 291745021
2019-11-14Allow all runtime tests for a language to be run via a single command.Kevin Krakauer
This was intended behavior per the README, but running tests without the --test flag caused an error. Users can now omit the --test flag to run every test for a runtime. PiperOrigin-RevId: 280522025
2019-11-01Add "manual" tag back to runtime tests.Nicolas Lacasse
PiperOrigin-RevId: 277971910
2019-10-30Add Kokoro config for new runtime testsBrad Burlage
PiperOrigin-RevId: 277607217
2019-10-09Add blacklists for remaining runtime tests, and test that they parse correctly.Nicolas Lacasse
PiperOrigin-RevId: 273781112
2019-10-01Add blacklist support to the runtime test runner.Nicolas Lacasse
Tests in the blacklist will be explicitly skipped (with associated log line). Checks in a blacklist for the nodejs tests. PiperOrigin-RevId: 272272749
2019-09-23Run all runtime tests in a single container.Nicolas Lacasse
This makes them run much faster. Also cleaned up the log reporting. PiperOrigin-RevId: 270799808
2019-09-18Shard the runtime tests.Nicolas Lacasse
Default of 20 shards was arbitrary and will need fine-tuning in later CLs. PiperOrigin-RevId: 269922871
2019-09-17Follow-up fixes for image tests.Nicolas Lacasse
- Fix ARG syntax in Dockerfiles. - Fix curl commands in Dockerfiles. - Fix some paths in proctor binaries. - Check error from Walk in search helper. PiperOrigin-RevId: 269641686
2019-09-16Refactor and clean up image tests.Nicolas Lacasse
* Use multi-stage builds in Dockerfiles. * Combine all proctor binaries into a single binary. * Change the TestRunner interface to reduce code duplication. PiperOrigin-RevId: 269462101
2019-09-03Impose order on test scripts.Adin Scannell
The simple test script has gotten out of control. Shard this script into different pieces and attempt to impose order on overall test structure. This change helps lay some of the foundations for future improvements. * The runsc/test directories are moved into just test/. * The runsc/test/testutil package is split into logical pieces. * The scripts/ directory contains new top-level targets. * Each test is now responsible for building targets it requires. * The install functionality is moved into `runsc` itself for simplicity. * The existing kokoro run_tests.sh file now just calls all (can be split). After this change is merged, I will create multiple distinct workflows for Kokoro, one for each of the scripts currently targeted by `run_tests.sh` today, which should dramatically reduce the time-to-run for the Kokoro tests, and provides a better foundation for further improvements to the infrastructure. PiperOrigin-RevId: 267081397
2019-08-29Compile procter binaries during image creation.Nicolas Lacasse
Using "go run ..." in the ENTRYPOINT causes the go compiler to run each time the container is started. We can just compile the binary once as part of the image. PiperOrigin-RevId: 266212462
2019-08-20Don't run runtime tests in parallel.Nicolas Lacasse
We need real sharding, and will let Bazel handle the parallelization. That is coming soon. Until then, remove this call to t.Parallel() so that we can run the tests without eating all CPU. PiperOrigin-RevId: 264498919
2019-08-09Create tests for common.Search().Brett Landau
Using the path_test.go file built by the Golang devs as a base, tests have been created to verify the functionality of common.Search(). A mock file system is created and fake test files are generated to see if they get picked up by common.Search(). Also included in this CL is a bug fix for proctor-nodejs that was discovered using this test. proctor-nodejs used to allow multiple "-" in its test name filter. The regex has been updated to prevent this. PiperOrigin-RevId: 262647263
2019-08-05Alter Dockerfiles to include common.go and use a prebuilt JDK.Samantha Sample
After the refactoring of the proctor binaries, the Dockerfiles for each language must be altered to copy the common folder into their image. Additionally, Java has been changed to use the pre-built version of JDK-11 from Ubuntu, instead of building it from the source. This allows for a smaller image and faster test execution within the container. PiperOrigin-RevId: 261805158
2019-08-05Expand runtimes test suite to include Go, Java, PHP, and Python.Samantha Sample
This change adds functionality for running more languages using the runtimes test suite. It divides the languages into separate test functions, which each call the helper testLang function in the runtimes_test.go file. This allows them to be run individually or as a group. PiperOrigin-RevId: 261791935
2019-08-01Refactor ListTests() to common.Search().Brett Landau
This change removes the filepath.Walk() function from proctor- go, php, and nodejs. The filepath.Walk() is now defined in common.go in Search(). Each proctor binary passes root directory and testFilter arguments to Search(). proctor-python.go no longer uses filepath.Walk() to search for tests. There is a built-in list test function within python's language test suite so that is being used instead. PiperOrigin-RevId: 261242897
2019-07-31Fix test execution bugs in proctor-go and proctor-python.Brett Landau
proctor-go had a bug where it would incorrectly identify a tool test as a disk test. Instead of searching for the test on disk as the identification method, we now check if the test name ends in ".go". If the test ends in ".go" it is run as a disk test, otherwise the test is run as a tool test. Python tests need to be run from within the directory they exist. Functionality to split the test name from it's parent directory has been added and a cmd.Dir argument has been set. PiperOrigin-RevId: 261021693
2019-07-31Refactor proctor binaries to implement testRunner interface.Brett Landau
Shared code among proctor-*.go files has been refactored into common/common.go. The common package is imported in each proctor binary and a struct is created to implement the testRunner interface defined in common.go. This allows for the proctor binaries to be updated without having to copy/paste the same code across all files. There are no usage or functionality changes. PiperOrigin-RevId: 260967080
2019-07-29Move runtimes tests to appropriate directory.Samantha Sample
PiperOrigin-RevId: 260577765