add Elastic Stack and Elastic Package version to test results report

[teresaromero]

Resolves #991

Elastic stack version and elastic-package running version are now logged into the running test output when elastic-stack test is run.

As per requested in comments, it also introduces a new json report output format.

[teresaromero]

@efd6 should this information be across all the tests? (static, asset, pipeline, etc...)

Let me know if the elastic-package version format is ok or you have any other thought

[efd6]

AFIUI the version is a constant over the table, correct? If that's the case, the additional verbosity is not a win.

In the output of benchstat for example, the constant data is provide as a preamble to the table.

$ benchstat new.bench old.bench 
goos: darwin
goarch: amd64
pkg: github.com/elastic/go-libaudit/v2/aucoalesce
cpu: Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
                    │  new.bench  │             old.bench              │
                    │   sec/op    │   sec/op     vs base               │
CoalesceMessages-16   4.191µ ± 3%   4.594µ ± 4%  +9.62% (p=0.000 n=10)

                    │  new.bench   │              old.bench              │
                    │     B/op     │     B/op      vs base               │
CoalesceMessages-16   2.753Ki ± 0%   2.786Ki ± 0%  +1.21% (p=0.000 n=10)

                    │ new.bench  │             old.bench             │
                    │ allocs/op  │ allocs/op   vs base               │
CoalesceMessages-16   13.00 ± 8%   14.00 ± 0%  +7.69% (p=0.003 n=10)

In terms of other thought, I'd love it if the tool emitted JSON instead of a table (or at least optionally instead of), since then any format can by chosen by the user after processing the machine-readable format.

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: c0d5ac6

History

• 💚 Build #6572 succeeded c580a91

[teresaromero]

AFIUI the version is a constant over the table, correct? If that's the case, the additional verbosity is not a win.

As you said, its not a result perse, but a constant defining the test. I've added as a command log when running each test. To be part of the table/xml i guess it should be part of the result and redundant to each test case.

In terms of other thought, I'd love it if the tool emitted JSON instead of a table (or at least optionally instead of), since then any format can by chosen by the user after processing the machine-readable format.

i've added the option to make a json file/output with the test results