Multiple spec test failures - RCA required

[sanfrancrisko]

There are a number of test failures when trying to run the spec tests for the project whilst pointing to the latest commit on main in puppetlabs/puppet

export PUPPET_GEM_VERSION=https://github.com/puppetlabs/puppet.git#main 
bundle install
bundle exec rake test:spec

rspec ./spec/applications/orch_app_spec.rb:17 # orch_app with params is expected to compile into a catalogue without dependency cycles
rspec ./spec/applications/orch_app_spec.rb:18 # orch_app with params is expected to contain Orch_app[my_awesome_app]
rspec ./spec/applications/orch_app_spec.rb:19 # orch_app with params is expected to contain Orch_app::Db[my_awesome_app]
rspec ./spec/classes/test_registry_spec.rb:10 # test::registry is expected to compile into a catalogue without dependency cycles
rspec ./spec/defines/sysctl_spec.rb:12 # sysctl is expected to contain Class[sysctl::common]
rspec ./spec/defines/sysctl_spec.rb:13 # sysctl is expected to contain Augeas[sysctl/vm.swappiness] with context => "/files/etc/sysctl.conf", changes => "set vm.swappiness '60'", changes => #<Proc:0x00007fdcb7bc1538@/Users/ciaran.mccrisken/sanfrancrisko/rspec-puppet/spec/defines/sysctl_spec.rb:3>, onlyif => "match vm.swappiness[.='60'] size == 0", notify => "Exec[sysctl/reload]" and foo undefined
rspec ./spec/defines/sysctl_spec.rb:20 # sysctl is expected to contain exactly 1 Sysctl resource
rspec ./spec/defines/test_api_spec.rb:8 # sysctl rspec group should have a catalogue method
rspec ./spec/defines/test_api_spec.rb:12 # sysctl rspec group subject should return a catalogue
rspec ./spec/defines/test_api_spec.rb:16 # sysctl rspec group should be included in the coverage filter
rspec ./spec/functions/ensure_packages_spec.rb:6 # ensure_packages should create the resource in the catalogue

Whilst keeping rspec-puppet at HEAD of master, I've tried working back through previous Puppet versions to try and determine if any changes in the puppetlabs/puppet repo could be identified as a cause. Given we have a successful run of 6.20.0 here, it seems to be suggesting it's probably a combination of something on rodjkek/rspec-puppet#master and puppetlabs/puppet#[master|main].

Following that thought, after attempting to run tests with v2.8.0 of rspec-puppet against v6.20.0 of puppet, the following failures emerged:

rspec ./spec/classes/test_registry_spec.rb:10 # test::registry is expected to compile into a catalogue without dependency cycles
rspec ./spec/defines/sysctl_spec.rb:12 # sysctl is expected to contain Class[sysctl::common]
rspec ./spec/defines/sysctl_spec.rb:13 # sysctl is expected to contain Augeas[sysctl/vm.swappiness] with context => "/files/etc/sysctl.conf", changes => "set vm.swappiness '60'", changes => #<Proc:0x00007faa08949940@/Users/ciaran.mccrisken/sanfrancrisko/rspec-puppet/spec/defines/sysctl_spec.rb:3>, onlyif => "match vm.swappiness[.='60'] size == 0", notify => "Exec[sysctl/reload]" and foo undefined
rspec ./spec/defines/sysctl_spec.rb:20 # sysctl is expected to contain exactly 1 Sysctl resource
rspec ./spec/defines/test_api_spec.rb:8 # sysctl rspec group should have a catalogue method
rspec ./spec/defines/test_api_spec.rb:12 # sysctl rspec group subject should return a catalogue
rspec ./spec/defines/test_api_spec.rb:16 # sysctl rspec group should be included in the coverage filter
rspec ./spec/functions/ensure_packages_spec.rb:6 # ensure_packages should create the resource in the catalogue

There's more investigation required here to try and determine where the failures are manifesting - certainly more work than originally anticipated, hence logging this ticket.