support a better puppet python installer

[binary1230]

currently puppet doesn't know how to detect when we don't need to re-run the install scripts because it doesn't know how to intelligently install the python packages, so it re-installs everthing every single run.

most of the time we don't ever need to re-install dependencies, so fixing this would help take deploy times down to a matter of seconds instead of minutes.

perhaps investigate this:
https://github.com/stankevich/puppet-python

[binary1230]

https://github.com/stankevich/puppet-python

might try using something in there like:

python::requirements { '/var/www/project1/requirements.txt' :
virtualenv => '/var/www/project1',
proxy => 'http://proxy.domain.com:3128',
owner => 'appuser',
group => 'apps',
}

@EliAndrewC where did the setup.py script come from, is that a homegrown thing? In theory do you think it should be possible to replace it with it's puppet equivalent?

[EliAndrewC]

Are you sure that the Python setup.py is actually reinstalling everything on every run? It really shouldn't be; setup.py is a standard Python convention which is part of the official setuptools ecosystem, and it should check every dependency, see whether we have it installed, and not reinstall if it's already there.

Note that running setup.py develop will still print a lot of messages to the screen for all of the packages, but if you look carefully I think you'll see that it's just saying the dependency is already satisfied.

[binary1230]

oh uh didn't realize that, I'll check.

in general, the entire puppet everything needs to be revisited.

what would be nice to be able to get though is if anything changed. maybe returning a non-zero error code if

i.e. I'd like to detect, in puppet, the situation where we upgrade a package version in requirements.txt and install it. if and only that event happens, then we need to do the rest of the installation process, if not, we can skip to later steps in the process.

everytime we run puppet now, the current puppet manifests run every step of the process, which just wastes time with deployments. also, eventually if this was working, we can do the concept of a 'dry run' and let puppet tell us the specific list of things it needs to do in order to deploy the latest release, including 'do nothing'

[binary1230]

*entire puppet everything -dependency- related needs to be revisited

[EliAndrewC]

That's a good point; being able to run a command that says "do I need to do anything on this system to make it current" would be pretty useful.