Simplify profiling and benchmarks.

- Tweak the benchmarks to work with b.N where appropriate. In many cases,
  b.N was simply being ignored. This creates an implicit dependency in the
  user passing a reasonable benchtime (less than or equal to the actual
  runtime of the test, or using the X syntax) otherwise the test runs
  forever.
- In cases where the above is impossible, explicitly set benchtime from
  the test wrapper, to prevent the above behavior (tensorflow).
- Drop the *Reverse variants, which are simply hey benchmarks. We should
  just add a hey benchmark. The platforms benchmarks already include a
  native platform, and thus these benchmarks are incredibly confusing.
  (In other words, BenchmarkNginxReverse has nothing to do with an nginx
  benchmark for runsc.)
- Remove the redunant Harness object, which contains no state, in order
  to slightly simplify the code.
- Make Block and Heap profiling actually work, but setting appropriate
  runtime parameters (and plumbing them through the config).
- Split the profiling into two phases: start and stop, since some will
  need to be started early, and others will need to happen at the end.

PiperOrigin-RevId: 349495377

1 files changed, 10 insertions, 6 deletions

diff --git a/test/benchmarks/harness/harness.go b/test/benchmarks/harness/harness.go
index 4c6e724aa..a853b7ba8 100644
--- a/test/benchmarks/harness/harness.go
+++ b/test/benchmarks/harness/harness.go
@@ -28,12 +28,8 @@ var (
 	debug = flag.Bool("debug", false, "turns on debug messages for individual benchmarks")
 )
 
-// Harness is a handle for managing state in benchmark runs.
-type Harness struct {
-}
-
 // Init performs any harness initilialization before runs.
-func (h *Harness) Init() error {
+func Init() error {
 	flag.Usage = func() {
 		fmt.Fprintf(os.Stderr, "Usage: %s -- --test.bench=<regex>\n", os.Args[0])
 		flag.PrintDefaults()
@@ -47,7 +43,15 @@ func (h *Harness) Init() error {
 	return nil
 }
 
+// SetFixedBenchmarks causes all benchmarks to run once.
+//
+// This must be set if they cannot scale with N. Note that this uses 1ns
+// instead of 1x due to https://github.com/golang/go/issues/32051.
+func SetFixedBenchmarks() {
+	flag.Set("test.benchtime", "1ns")
+}
+
 // GetMachine returns this run's implementation of machine.
-func (h *Harness) GetMachine() (Machine, error) {
+func GetMachine() (Machine, error) {
 	return &localMachine{}, nil
 }