Multi-variant states #30
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This required some exploratory and context refreshing isolation tests for the test object parsing capabilities of the test loader. In particular, these tests were used to identify what is the best way to parametrically (for state backend access) track all the variants of the components of a net composite and by extension a test node. It also involved some docstring clarifications regarding the test object parsing interface. For all current state backend logging we will still rely on just the vm suffix instead of the whole variant name since the latter must be inferrable by means of the currently run tests.
A variant restriction parameter remains a parameter and as such must be allowed an empty value in order to drop any previous or default values (here variant restrictions). This includes some isolation test separation.
In particular, update the legacy Fedora 18 to Fedora 33 with a full DVD unattended install and variant definitions. As the added integration tests uncover multiple issues right now, we will have to fix each one next in isolation.
Previously it was possible that a restriction of one suffix variant for a vm like only_vm1 leaks into choices of vm2 variants by matching the wrong part of the net variant name and thus producing incorrect node variants as a result. The workaround for this was a rather repetitive only_vm1 = vm1.qemu_kvm_centos in order to match the right net and vm1 variant. Now the original configuration works properly only_vm1 = qemu_kvm_centos A second undetected problem of the previous implementation was that restriction of a given variant might also apply to another if the two are parsed separately. The reason was that a more restrictive initial test set would result in fewer (but not zero) vms and if these are then reused by a second test set, no new nets and vms will be parsed even if such are still needed. Solve both problems by completely rewriting the nets get and parse functionality with isolation tests that detect the above problems and improve the overall readability of the complex composite net retrieval for each node.
This handles cases where a conditional block restriction would prevent the conditional vm variant from composition with some other object variant. A conditional restriction like that would only take effect once the complete test node is parsed and can therefore only be applied at this stage.
A parent node (with multiple vm1 variants) of a node involving a single vm1 variant might result in duplication if parsed via another object like vm2. For example, tutorial_gui can work with both vm1.qemu_kvm_centos and vm1.qemu_kvm_fedora but their child nodes tutorial_get if restricted to just vm1.qemu_kvm_centos would still get cloned as tutorial_get..qemu_kvm_fedora due to two parent nodes via vm2 which were so far restricted only using vm2. The new approach will make sure to apply any knowledge of other test object variants if such are applicable for reducing the set of reusable parents, rendering such cloning impossible.
Everything beyond this tests complete graph parsing and traversal.
Limitations in the Cartesian config and other external factors imply that for the moment we will have settle with extra todos for needed fixes. Thankfully the current functionality is already good enough for most our use cases.
pevogam
force-pushed
the
multi-variant-states
branch
from
ef730f6 to
962c8b4
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This branch adds support for multi-variant external states (i.e.
external states of multi-variant stateful objects) improving on the
command line handling, isolation and integration testing, and
previous problems and limitations.