RFC: sysinfo collectibles dynamically defined per job

[narasimhan-v]

Similar to tags per test, can we have sysinfo (files, commands, profilers) defined per test, and added to the sysinfo files per job basis ?

Example:

:avocado: tags=cpu,privileged
:avocado: files=/proc/cpuinfo,/proc,softirqs
:avocado: commands=ppc64_cpu --smt

[beraldoleal]

@willianrampazzo not sure, but maybe this is related to the new assets architecture, right?

[willianrampazzo]

@willianrampazzo not sure, but maybe this is related to the new assets architecture, right?

I'm not sure. This seems a list of commands to be executed during a stage of the job, not a requirement that is needed but the test to run. We need to elaborate more on the idea to decide that.

[beraldoleal]

@willianrampazzo ok, not a problem. I thought that the requirements modules would include files in the system too.

[willianrampazzo]

@beraldoleal yes, it includes files, but as far as I could understand, the proposal here is to execute a list of commands during the sysinfo. The list of commands may fit into the requirements for the test, it is just a matter of specifying what is needed and how it can fit as a requirement for the resolver. The proposal here, for me, seems a little vague.

[narasimhan-v]

@willianrampazzo The idea here is to have sysinfo collectibles per test. We have many tests across categories, and the sysinfo for those are defined for all tests. We need a way to have specific sysinfo per test, and defining them in the test was something I could think of.

[narasimhan-v]

@willianrampazzo @beraldoleal @clebergnu This was dormant for a while now. Let me rephrase the requirement.

When we define the sysinfo collectibles, they are common for all tests. But we do not need them for all tests.
Each test can have its own list of needed sysinfo.

So, collect all needed sysinfo for all tests, with the current model, we are forced to do one of these 2:

1. Make a union of all sysinfo collectibles, and run all tests in a sequence.
2. Set sysinfo for a group of tests, run that group of tests with same collectibles, repeat this for other groups.

The proposal here is:
Have test define its own sysinfo collectibles.
So, when we run multiple tests, each test is run with its own defined sysinfo only collected.

How and where do we define it, is the next thing to decide, if this approach looks fine.

[willianrampazzo]

@narasimhan-v I got it now. I think having sysinfo per test is a valid use case. I like the approach of test docstrings, but it needs to be easy to use and easy to parse. This looks really similar to what I'm working in the Requirements Resolver, but with a different objective.

To make it easier to parse, I would go with something more specific. We could try the same proposed approach from the BP002 (https://avocado-framework.readthedocs.io/en/80.0/blueprints/BP002.html#requirements-representation-as-metadata-on-test-docstring) like:

:avocado: sysinfo={"type": "file", "uri": "/proc/cpuinfo"}
:avocado: sysinfo={"type": "file", "uri": "/proc/softirqs"}
:avocado: sysinfo={"type": "command", "uri": "/path/to/file/ppc64_cpu", "args": ["--smt"]}

The natural evolution could be to add support to other representations, just like in the BP002 for Requirements (https://avocado-framework.readthedocs.io/en/80.0/blueprints/BP002.html#requirements-representations).

[narasimhan-v]

@willianrampazzo Sure, that sounds interesting and way better to handle all requirements. How far along are we ? Is there any way I can also pitch in ?

[willianrampazzo]

@willianrampazzo Sure, that sounds interesting and way better to handle all requirements. How far along are we ? Is there any way I can also pitch in ?

What I mean was that the representation of the sysinfo could be the same as the requirements, but I don't think this would fit the requirements resolver. I think the sysinfo collection is related to pre and post execution of a test, while the requirements feature is a pre test task.

I can see both features implemented independently. The Requirements Resolver is on hold until we advance with the N(ext)Runner development, but I can see the development of sysinfo plugin in the way you need right now without any dependency.

[narasimhan-v]

@willianrampazzo I assume the implementation of the requirements is going to be independent of whether it is pre or post ? If so, why don't we have pre/post tag also part of the requirement and handle it wherever needed ?

[willianrampazzo]

@willianrampazzo I assume the implementation of the requirements is going to be independent of whether it is pre or post ? If so, why don't we have pre/post tag also part of the requirement and handle it wherever needed ?

I think that is possible. My suggestion to have your plugin developed now, independent of any other thing, is to speed up the availability, as the requirements resolver development is on hold for now.

[narasimhan-v]

Sure, got it @willianrampazzo