requirements: Add all mandatory requirements

[ldoktor]

The requirements are quite outdated, this commit adds all of them to
both, requirements.txt and setup.py in order to just install avocado-vt
and get all required dependencies installed automatically.

Signed-off-by: Lukáš Doktor [email protected]

[ldoktor]

OK, we have a problem, because avocado-36lts is called avocado while the newer Avocados are called avocado-framework. Let me think of a solution, @clebergnu any ideas?

[ldoktor]

avocado_vt: Add avocado (and other) requirements

[ldoktor]

@clebergnu would you please take a look at this one?

[clebergnu]

@ldoktor after successfully installing requirements on requirements.txt, this is what get when running this on a fresh environment:

Processing dependencies for avocado-plugins-vt==53.0
Searching for avocado-framework>=36.0
Reading https://pypi.python.org/simple/avocado-framework/
Downloading https://pypi.python.org/packages/ce/37/0b60e639fee3993fde67ae992e84d68b8266d322fd8807c7cbafe2abb696/avocado-framework-54.0.tar.gz#md5=28760d3dbde674bedf56e012f352c1c6
Best match: avocado-framework 54.0
Processing avocado-framework-54.0.tar.gz
Writing /tmp/easy_install-PscIBs/avocado-framework/setup.cfg
Running avocado-framework/setup.py -q bdist_egg --dist-dir /tmp/easy_install-PscIBs/avocado-framework/egg-dist-tmp-uS874q
warning: no files found matching 'examples/*.yaml'

Following this (put aside to aid the reading of the error itself):

error: Setup script exited with error: SandboxViolation: open('/etc/avocado/avocado.conf', 'wb') {}

The package setup script has attempted to modify files on your system
that are not within the EasyInstall build area, and has been aborted.

This package cannot be safely installed by EasyInstall, and may not
support alternate installation locations even if you run its setup
script by hand.  Please inform the package's author and the EasyInstall
maintainers to find out if a fix or workaround is available.

Running pip install avocado-framework works just fine, on the very same environment. So, it looks like the problem is having it chained inside another module (avocado-vt in this case) setup.py.

Maybe the easy solution is to have both a requirements.txt and a requirements-py26.txt file?

[clebergnu]

If the goal is to check for Avocado, the way to go is to use pkg_resources.

[ldoktor]

OK, I can change that.

[ldoktor]

Well, I could detect avocado-framwork and proceed, but in case it's not present, but avocado is we'd have to still check whether it is our avocado, or whether it's just some other avocado module (like the django one). Using this is IMO simpler and clearer, what do you think?

[ldoktor]

Or let's just simply remove all of it. It's <36 code...

[clebergnu]

"requires stevedore" or "requires the stevedore dependency"

[ldoktor]

ok

[ldoktor]

Can you please provide the actual steps? I can't reproduce your issue by:

$ virtualenv -p python venv
$ source venv/bin/activate
$ cd $avocado-vt
$ pip install -r requirements.txt  # fails with Autotest attempting to write /etc
$ cd $autotest
$ pip install .    # successful install (patches for venv were already merged)
$ cd $avocado-vt
$ pip install -r requirements.txt
$ pip install .
$ avocado plugins | grep vt
Failed to load plugin from module "avocado_vt.plugins.vt_list": EnvironmentError("Bootstrap missing. Execute 'avocado vt-bootstrap' or disable this plugin to get rid of this message",)
Failed to load plugin from module "avocado_vt.plugins.vt": EnvironmentError("Bootstrap missing. Execute 'avocado vt-bootstrap' or disable this plugin to get rid of this message",)
vt-bootstrap Avocado VT - implements the 'vt-bootstrap' subcommand
vt-joblock  Avocado-VT Job Lock/Unlock

# shorter version
$ virtualenv -p python venv
$ source venv/bin/activate
$ cd $autotest
$ pip install .
$ cd $avocado-vt
$ pip install .
$ avocado plugins | grep vt
Failed to load plugin from module "avocado_vt.plugins.vt_list": EnvironmentError("Bootstrap missing. Execute 'avocado vt-bootstrap' or disable this plugin to get rid of this message",)
Failed to load plugin from module "avocado_vt.plugins.vt": EnvironmentError("Bootstrap missing. Execute 'avocado vt-bootstrap' or disable this plugin to get rid of this message",)
vt-bootstrap Avocado VT - implements the 'vt-bootstrap' subcommand
vt-joblock  Avocado-VT Job Lock/Unlock

Both work fine.

[clebergnu]

@ldoktor the biggest difference is that I tried a system wide install, running as root. That's relevant also because it's similar to what RPM package build will try to do. Let me know if you can replicate that.

[ldoktor]

@clebergnu still unable to reproduce. Sharing the exact steps would help, this time I tried:

$ docker run -t -i --rm -v `pwd`:/avocado-vt fedora /bin/bash
$ dnf install # gcc, python-devel, ...
$ pip install .
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip install --user` instead.
Processing /avocado-vt
  Requirement already satisfied (use --upgrade to upgrade): avocado-plugins-vt==53.0 from file:///avocado-vt in /usr/lib/python2.7/site-packages
Collecting netifaces (from avocado-plugins-vt==53.0)
  Using cached netifaces-0.10.6.tar.gz
Collecting aexpect (from avocado-plugins-vt==53.0)
  Using cached aexpect-1.4.0.tar.gz
Collecting netaddr (from avocado-plugins-vt==53.0)
  Using cached netaddr-0.7.19-py2.py3-none-any.whl
Collecting simplejson (from avocado-plugins-vt==53.0)
  Using cached simplejson-3.11.1.tar.gz
Collecting autotest (from avocado-plugins-vt==53.0)
  Using cached autotest-0.16.4.tar.gz
Collecting avocado-framework>=36.0 (from avocado-plugins-vt==53.0)
  Using cached avocado-framework-54.0.tar.gz
Collecting stevedore>=0.14 (from avocado-framework>=36.0->avocado-plugins-vt==53.0)
  Using cached stevedore-1.27.1-py2.py3-none-any.whl
Collecting six>=1.9.0 (from stevedore>=0.14->avocado-framework>=36.0->avocado-plugins-vt==53.0)
  Using cached six-1.11.0-py2.py3-none-any.whl
Collecting pbr!=2.1.0,>=2.0.0 (from stevedore>=0.14->avocado-framework>=36.0->avocado-plugins-vt==53.0)
  Using cached pbr-3.1.1-py2.py3-none-any.whl
Installing collected packages: netifaces, aexpect, netaddr, simplejson, autotest, six, pbr, stevedore, avocado-framework
  Running setup.py install for netifaces ... done
  Running setup.py install for aexpect ... done
  Running setup.py install for simplejson ... done
  Running setup.py install for autotest ... done
  Running setup.py install for avocado-framework ... done
Successfully installed aexpect-1.4.0 autotest-0.16.4 avocado-framework-54.0 netaddr-0.7.19 netifaces-0.10.6 pbr-3.1.1 simplejson-3.11.1 six-1.11.0 stevedore-1.27.1
avocado plugins
Failed to load plugin from module "avocado_vt.plugins.vt_list": EnvironmentError("Bootstrap missing. Execute 'avocado vt-bootstrap' or disable this plugin to get rid of this message",)
Failed to load plugin from module "avocado_vt.plugins.vt": EnvironmentError("Bootstrap missing. Execute 'avocado vt-bootstrap' or disable this plugin to get rid of this message",)
Plugins that add new commands (cli.cmd):
config       Shows avocado config keys
diff         Shows the difference between 2 jobs.
distro       Shows detected Linux distribution
exec-path    Returns path to avocado bash libraries and exits.
list         List available tests
multiplex    Tool to analyze and visualize test variants and params
plugins      Displays plugin information
run          Runs one or more tests (native test, test alias, binary or script)
sysinfo      Collect system information
variants     Tool to analyze and visualize test variants and params
vt-bootstrap Avocado VT - implements the 'vt-bootstrap' subcommand

Plugins that add new options to commands (cli):
envkeep     Keep variables in remote environment
gdb         GDB options for the 'run' subcommand
journal     Journal options for the 'run' subcommand
json        JSON output options for 'run' command
replay      Replay options for 'run' subcommand
tap         TAP - Test Anything Protocol results
wrapper     Implements the '--wrapper' flag for the 'run' subcommand
xunit       xUnit output options
zip_archive Result archive (ZIP) support to run command

Plugins that run before/after the execution of jobs (job.prepost):
human       Human Interface UI
jobscripts  Runs scripts before/after the job is run
teststmpdir Creates a temporary directory for tests consumption
vt-joblock  Avocado-VT Job Lock/Unlock

Plugins that generate job result in different formats (result):
json        JSON result support
xunit       XUnit result support
zip_archive Result archive (ZIP) support

Plugins that generate job result based on job/test events (result_events):
human   Human Interface UI
journal Journal event based results implementation
tap     TAP - Test Anything Protocol results

Plugins that generate test variants (varianter): 
(No active plugin)

[ldoktor]

OK, I'm able to reproduce it with python setup.py install (python setup.py -q bdist_egg --dist-dir /tmp/avocado-framework/egg-dist-tmp-5MfHyp), but it is Avocado issue, not this PR issue. Anyway working on a fix.

[ldoktor]

@clebergnu This is not intended for merging right now, but it's WiP to tackle the packaging of Avocado-vt. I'm leaving on vacation so feel free to think about this and let me know. I still have to make it work with LTS (by making it optional) and move the rest of the package_data into avocado_vt (or virttest), but generally it should cover most of the things mentioned in the card as well as in @lmr's #1602.

Currently this version requires avocado-framework/avocado#2780 (but the real version should not require it to support 52 LTS)

[ldoktor]

Changes

• requirements: Add all mandatory requirements - reworked to include only mandatory requirements and use optional ones in requirements.txt
• requirements: Add all mandatory requirements - updated versions
• debian: Update distribution name - changed location
• Use SettingPlugin to register config files - fixed locations and set requirements to include only supported versions.

[lgtm-com]

This pull request introduces 8 alerts and fixes 1 when merging f466319 into c7143ef - view on LGTM.com

new alerts:

• 4 for Unused exception object
• 2 for Missing call to __init__ during object initialization
• 1 for Unused import
• 1 for Module is imported with 'import' and 'import from'

fixed alerts:

• 1 for Signature mismatch in overriding method

[lgtm-com]

This pull request introduces 7 alerts and fixes 1 when merging 0c6d354 into c7143ef - view on LGTM.com

new alerts:

• 4 for Unused exception object
• 2 for Missing call to __init__ during object initialization
• 1 for Module is imported with 'import' and 'import from'

fixed alerts:

• 1 for Signature mismatch in overriding method

[ldoktor]

Oups, forgot to include rpm changes and also I want to reorder to configs (insert them to 0 instead of extending). Will send it later today...

[ldoktor]

Well, we still need to deliver certain files outside the package, which is not pythonic, but hopefully fix for that can wait for next-time as if I understand it correctly it requires to deliver virttest separately and also to package all shared files in virttest module and only deploy it system-wide using rpm.

Anyway, new changes:

• setup.py: Standardize plugin naming - Fixed RPM spec file
• Use SettingPlugin to register config files - Included RPM spec file changes
• Use SettingPlugin to register config files - Insert the config files, instead of append (this should be closer to previous behaviour, because this way the packaged config should be least-significant and /etc as well as user config should override it)

[lgtm-com]

This pull request introduces 7 alerts and fixes 1 when merging 5ffef81 into 3001176 - view on LGTM.com

new alerts:

• 4 for Unused exception object
• 2 for Missing call to __init__ during object initialization
• 1 for Module is imported with 'import' and 'import from'

fixed alerts:

• 1 for Signature mismatch in overriding method

[clebergnu]

I've tested this extensively, and given that it affects aspects that would need to be dealt in the release process anyway, I'm merging these as a build/release fix. Thanks!