summaryrefslogtreecommitdiffhomepage
path: root/test/benchmarks
AgeCommit message (Collapse)Author
2021-08-16Internal change.gVisor bot
PiperOrigin-RevId: 391217339
2021-08-13[benchmarks] Update BenchmarkStartEmpty benchmark.Zach Koopmans
Update the start benchmark on empty to only "Start" a container, not wait for its completion. TL:DR only measure the actual start call for the empty container. Previously, we were measuring the completion of /bin/true in container alpine AND the cleanup. This was fine until profiling started failing all the time on ptrace. This is a cost that runc is not paying. These changes will reduce the over all timing of the benchmark, but it will give more sane results. Instead, use "Spawn" which is similar to `docker run --detach alpine /bin/sleep 100`. Call sleep so containers stick around long enough for the profiler to read profile data from them. PiperOrigin-RevId: 390705431
2021-06-11[benchmarks] Fix redis benchmark.Zach Koopmans
Intermittenly, the connection between the client redis_benchmark container can be flaky with the server, even if the server has been up for a long time. If this happens, just re-run a client until we get a result. Also, don't start a new server for each operation. Also also, modify Makefile run-benchmark call to accept RUNTIME=runc correctly. PiperOrigin-RevId: 378918886
2021-04-03Remove eternal and enormous tests.Adin Scannell
PiperOrigin-RevId: 366573366
2021-03-18Address post submit comments for fs benchmarks.Zach Koopmans
Also, drop fio total reads/writes to 1GB as 10GB is prohibitively slow. PiperOrigin-RevId: 363714060
2021-03-10Add a rootfs filesystem to fs benchmarks. Also, fix fio benchmark.Zach Koopmans
The previous "bind" filesystem, already included in go/runsc-benchmarks is a remote re-validate mount. However, the non-re-validate mount was not present, and it has been added in the form of rootfs. Also, fix the fio runs to reads/writes of 10GB as running with --test.benchtime=Xs may scale beyond the memory available to tmpfs mounts on buildkite VMs. Currently, our buildkite pipelines are run on e2-standard-8 machines with 32GB of memory, allowing tmpfs mounts to safely be at least 10GB. PiperOrigin-RevId: 362143620
2021-03-06[op] Replace syscall package usage with golang.org/x/sys/unix in test/.Ayush Ranjan
The syscall package has been deprecated in favor of golang.org/x/sys. Note that syscall is still used in some places because the following don't seem to have an equivalent in unix package: - syscall.SysProcIDMap - syscall.Credential Updates #214 PiperOrigin-RevId: 361332034
2021-03-02[rack] Support running tcp_benchmarks with RACK.Ayush Ranjan
PiperOrigin-RevId: 360491700
2021-02-19Correctly set and respect b.N in fio benchmark.Zach Koopmans
fio should scale by written/read bytes and not iterate runs of the fio container. PiperOrigin-RevId: 358511771
2021-02-18Make b.N increase by KB not bytes on iperf.Zach Koopmans
Currently, iperf runs a client that scales by bytes sent. In practice, this causes b.N to scale slowly and have several short lived containers. Instead, scale by KB to more quickly reach required time. PiperOrigin-RevId: 358244926
2021-02-12Rename params for iperf and tensorflow.Zach Koopmans
Rename operation params in iperf and tensorflow to match other similar benchmarks. PiperOrigin-RevId: 357250304
2021-02-12Fix bug with iperf and don't profile runc.Zach Koopmans
Fix issue with iperf where b.N wasn't changing across runs. Also, if the given runtime is runc/not given, don't run a profile against it. PiperOrigin-RevId: 357231450
2021-02-10Update benchmarks README.mdZach Koopmans
PiperOrigin-RevId: 356843249
2021-01-13Fix minor typosMichael Pratt
PiperOrigin-RevId: 351606635
2021-01-12Fix simple mistakes identified by goreportcard.Adin Scannell
These are primarily simplification and lint mistakes. However, minor fixes are also included and tests added where appropriate. PiperOrigin-RevId: 351425971
2021-01-07Fix native benchmarks.Adin Scannell
PiperOrigin-RevId: 350509137
2021-01-05Add benchmarks targets to BuildKite.Adin Scannell
This includes minor fix-ups: * Handle SIGTERM in runsc debug, to exit gracefully. * Fix cmd.debug.go opening all profiles as RDONLY. * Fix the test name in fio_test.go, and encode the block size in the test. PiperOrigin-RevId: 350205718
2020-12-29Simplify profiling and benchmarks.Adin Scannell
- Tweak the benchmarks to work with b.N where appropriate. In many cases, b.N was simply being ignored. This creates an implicit dependency in the user passing a reasonable benchtime (less than or equal to the actual runtime of the test, or using the X syntax) otherwise the test runs forever. - In cases where the above is impossible, explicitly set benchtime from the test wrapper, to prevent the above behavior (tensorflow). - Drop the *Reverse variants, which are simply hey benchmarks. We should just add a hey benchmark. The platforms benchmarks already include a native platform, and thus these benchmarks are incredibly confusing. (In other words, BenchmarkNginxReverse has nothing to do with an nginx benchmark for runsc.) - Remove the redunant Harness object, which contains no state, in order to slightly simplify the code. - Make Block and Heap profiling actually work, but setting appropriate runtime parameters (and plumbing them through the config). - Split the profiling into two phases: start and stop, since some will need to be started early, and others will need to happen at the end. PiperOrigin-RevId: 349495377
2020-12-11Fix long running bazel build jobs.Zach Koopmans
- Skip the bazel clean command on the last run of the benchmark. - Use --test.benchtime=1ns to force running the benchmark once (https://github.com/golang/go/issues/32051) PiperOrigin-RevId: 347124606
2020-12-11Adjust requests to be constant equal to b.N in network tests.Zach Koopmans
For "hey", requests >= concurrency. b.N can be set by the --test.benchtime={b.N}x. The previous setting of b.N * c can be surprisingly slow for larger c. Set the requests to max(b.N, c) and log to the user if it is c. PiperOrigin-RevId: 347053675
2020-12-09Add network benchmarks jobsZach Koopmans
Add httpd, nginx, node, and ruby benchmarks to continuous jobs. PiperOrigin-RevId: 346629115
2020-12-09Add tensorflow, ffmpeg, and redis jobs.Zach Koopmans
PiperOrigin-RevId: 346603153
2020-12-07Fix tags on benchmark targets.Zach Koopmans
PiperOrigin-RevId: 346203209
2020-11-30Do not os.Exit() from test/benchmarks/harness.Harness.Init with no args.Jamie Liu
PiperOrigin-RevId: 344896991
2020-11-16Remove ARP address workaroundGhanan Gowripalan
- Make AddressableEndpoint optional for NetworkEndpoint. Not all NetworkEndpoints need to support addressing (e.g. ARP), so AddressableEndpoint should only be implemented for protocols that support addressing such as IPv4 and IPv6. With this change, tcpip.ErrNotSupported will be returned by the stack when attempting to modify addresses on a network endpoint that does not support addressing. Now that packets are fully handled at the network layer, and (with this change) addresses are optional for network endpoints, we no longer need the workaround for ARP where a fake ARP address was added to each NIC that performs ARP so that packets would be delivered to the ARP layer. PiperOrigin-RevId: 342722547
2020-11-10Add debug logs to startup benchmark.Zach Koopmans
PiperOrigin-RevId: 341757694
2020-11-10Add all base and fs tests to Continuous Tests.Zach Koopmans
PiperOrigin-RevId: 341660511
2020-11-06Change size of startup benchmark and lower number of startups.Zach Koopmans
PiperOrigin-RevId: 341155693
2020-10-09Add parsers golang benchmarks.Zach Koopmans
Add parser and formatting for golang benchmarks for docker benchmarks. Change adds a library for printing and parsing Test parameters and metrics. Benchmarks use the library to print parameters in the Benchmark title (e.g. the name field in b.Run()), and to report CustomMetrics. Parser uses the library to parse printed data from benchmark output and put it into BigQuery structs. PiperOrigin-RevId: 336365628
2020-09-28Support creating protocol instances with Stack refGhanan Gowripalan
Network or transport protocols may want to reach the stack. Support this by letting the stack create the protocol instances so it can pass a reference to itself at protocol creation time. Note, protocols do not yet use the stack in this CL but later CLs will make use of the stack from protocols. PiperOrigin-RevId: 334260210
2020-09-24Fix Nginx Startup and Size Benchmarks.Zach Koopmans
Changes in Nginx Benchmarks in network_tests also affect Startup/Size Nginx Benchmarks. Make sure the commands line up. PiperOrigin-RevId: 333543697
2020-09-08Improve type safety for transport protocol optionsGhanan Gowripalan
The existing implementation for TransportProtocol.{Set}Option take arguments of an empty interface type which all types (implicitly) implement; any type may be passed to the functions. This change introduces marker interfaces for transport protocol options that may be set or queried which transport protocol option types implement to ensure that invalid types are caught at compile time. Different interfaces are used to allow the compiler to enforce read-only or set-only socket options. RELNOTES: n/a PiperOrigin-RevId: 330559811
2020-09-02Add Docs to nginx benchmark.Zach Koopmans
Adds docs to nginx and refactors both Httpd and Nginx benchmarks. Key changes: - Add docs and make nginx tests the same as httpd (reverse, all docs, etc.). - Make requests scale on c * b.N -> a request per thread. This works well with both --test.benchtime=10m (do a run that lasts at least 10m) and --test.benchtime=10x (do b.N = 10). -- Remove a doc from both tests (1000Kb) as 1024Kb exists. PiperOrigin-RevId: 329751091
2020-08-31Fix bug in bazel build benchmark.Zach Koopmans
PiperOrigin-RevId: 329409802
2020-08-25Add nogo support to go_binary and go_test targets.Adin Scannell
Updates #3374 PiperOrigin-RevId: 328378700
2020-08-10Add benchmarks to continuous build.Zach Koopmans
PiperOrigin-RevId: 325892974
2020-08-07Remove old benchmark tools.Zach Koopmans
Remove the old benchmark-tools directory, including imports in the WORKSPACE file and associated bazel rules. The new Golang benchmark-tools can be found at //test/benchmarks and it is functionally equivalent, excepting syscall_test which can be found in //test/perf/linux. PiperOrigin-RevId: 325529075
2020-08-07Port Ruby benchmark.Zach Koopmans
PiperOrigin-RevId: 325500772
2020-08-07Port Startup and Density Benchmarks.Zach Koopmans
PiperOrigin-RevId: 325497346
2020-08-04Port sysbench benchmark.Zach Koopmans
PiperOrigin-RevId: 324918229
2020-08-04Fix broken httpd_test.Bhasker Hariharan
PiperOrigin-RevId: 324822613
2020-08-03Add support for a reverse HTTPD test.Bhasker Hariharan
This change adds a new reverse HTTP test where the HTTPD server runs in a native container but the client runs inside gVisor. It allows us to test download performance under varying levels of concurrency. Also tweaks the concurrent request numbers to test for high levels of concurrency. PiperOrigin-RevId: 324651203
2020-07-30Port nginx and move parsers to own package.Zach Koopmans
This change: - Ports the nginx benchmark. - Switches the Httpd benchmark to use 'hey' as a client. - Moves all parsers to their own package 'tools'. Parsers are moved to their own package because 1) parsing output of a command is often dependent on the format of the command (e.g. 'fio --json'), 2) to enable easier reuse, and 3) clean up and simplify actual running benchmarks (no TestParser functions and ugly sample output in benchmark files). PiperOrigin-RevId: 324144165
2020-07-30Add runsc build benchmark.Zach Koopmans
PiperOrigin-RevId: 324071377
2020-07-30Update call in Node benchmark.Zach Koopmans
PiperOrigin-RevId: 324028183
2020-07-29Port fio benchmarkZach Koopmans
PiperOrigin-RevId: 323810654
2020-07-29Port node benchmark.Zach Koopmans
PiperOrigin-RevId: 323810235
2020-07-28Port tensorflow benchmark.Zach Koopmans
PiperOrigin-RevId: 323633737
2020-07-27Port ffmpeg benchmarkZach Koopmans
PiperOrigin-RevId: 323383320
2020-07-27Port redis benchmarkZach Koopmans
PiperOrigin-RevId: 323381964
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-12 18:01:46 +0000