Simplify profiling and benchmarks.

- Tweak the benchmarks to work with b.N where appropriate. In many cases,
  b.N was simply being ignored. This creates an implicit dependency in the
  user passing a reasonable benchtime (less than or equal to the actual
  runtime of the test, or using the X syntax) otherwise the test runs
  forever.
- In cases where the above is impossible, explicitly set benchtime from
  the test wrapper, to prevent the above behavior (tensorflow).
- Drop the *Reverse variants, which are simply hey benchmarks. We should
  just add a hey benchmark. The platforms benchmarks already include a
  native platform, and thus these benchmarks are incredibly confusing.
  (In other words, BenchmarkNginxReverse has nothing to do with an nginx
  benchmark for runsc.)
- Remove the redunant Harness object, which contains no state, in order
  to slightly simplify the code.
- Make Block and Heap profiling actually work, but setting appropriate
  runtime parameters (and plumbing them through the config).
- Split the profiling into two phases: start and stop, since some will
  need to be started early, and others will need to happen at the end.

PiperOrigin-RevId: 349495377

1 files changed, 4 insertions, 10 deletions

diff --git a/test/benchmarks/README.md b/test/benchmarks/README.md
index d1bbabf6f..1bfb4a129 100644
--- a/test/benchmarks/README.md
+++ b/test/benchmarks/README.md
@@ -81,11 +81,8 @@ benchmarks.
 In general, benchmarks should look like this:
 
 ```golang
-
-var h harness.Harness
-
 func BenchmarkMyCoolOne(b *testing.B) {
-  machine, err := h.GetMachine()
+  machine, err := harness.GetMachine()
   // check err
   defer machine.CleanUp()
 
@@ -95,14 +92,14 @@ func BenchmarkMyCoolOne(b *testing.B) {
 
   b.ResetTimer()
 
-  //Respect b.N.
+  // Respect b.N.
   for i := 0; i < b.N; i++ {
     out, err := container.Run(ctx, dockerutil.RunOpts{
       Image: "benchmarks/my-cool-image",
       Env: []string{"MY_VAR=awesome"},
       other options...see dockerutil
     }, "sh", "-c", "echo MY_VAR")
-    //check err
+    // check err...
     b.StopTimer()
 
     // Do parsing and reporting outside of the timer.
@@ -114,16 +111,13 @@ func BenchmarkMyCoolOne(b *testing.B) {
 }
 
 func TestMain(m *testing.M) {
-    h.Init()
+    harness.Init()
     os.Exit(m.Run())
 }
 ```
 
 Some notes on the above:
 
-*   The harness is initiated in the TestMain method and made global to test
-    module. The harness will handle any presetup that needs to happen with
-    flags, remote virtual machines (eventually), and other services.
 *   Respect `b.N` in that users of the benchmark may want to "run for an hour"
     or something of the sort.
 *   Use the `b.ReportMetric()` method to report custom metrics.