[WIP] Add dependency check for modules/plugins

[frennkie]

This PR adds the possibility to add the following dependencies to Modules:

• min_python_version (default 2.7) - the minimum version of Python that is required
• dependency_list_python (default: []) - a list of python packages that this Module needs
• dependency_list_system (default: []) - a list of system binaries that this Module needs

ToDos

• rebase (?!)
• complete dependencies
• update docs

[codecov-io]

Codecov Report

Merging #558 into master will decrease coverage by 0.35%.
The diff coverage is 72.61%.

@@            Coverage Diff             @@
##           master     #558      +/-   ##
==========================================
- Coverage   38.64%   38.28%   -0.36%     
==========================================
  Files          82       82              
  Lines       10081    10157      +76     
==========================================
- Hits         3896     3889       -7     
- Misses       6185     6268      +83

Impacted Files	Coverage Δ
viper/modules/misp.py	15.36% <100%> (-6.67%)	⬇️
viper/modules/exif.py	37.83% <100%> (+3.55%)	⬆️
viper/core/plugins.py	90.9% <100%> (+3.4%)	⬆️
viper/core/ui/commands.py	68.11% <20%> (-1.15%)	⬇️
viper/common/abstracts.py	87.37% <84.78%> (-2.1%)	⬇️
viper/core/session.py	73.8% <0%> (-1.2%)	⬇️
viper/common/out.py	85.88% <0%> (-1.18%)	⬇️
viper/core/ui/console.py	65.73% <0%> (-0.57%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8d7ad1f...53f98cd. Read the comment docs.

[frennkie]

         _
        (_)
   _   _ _ ____  _____  ____
  | | | | |  _ \| ___ |/ ___)
   \ V /| | |_| | ____| |
    \_/ |_|  __/|_____)_| v1.3-dev
          |_|

You have 2 files in your default repository

viper > about -h
usage: about [-h] [-m]

Show information about this Viper instance

optional arguments:
  -h, --help     show this help message and exit
  -m, --modules  Show info about modules

viper > about -m
+----------------+-------------------------------------------------+
| About          |                                                 |
+----------------+-------------------------------------------------+
| Viper Version  | 1.3-dev                                         |
| Python Version | 3.5.3                                           |
| Homepage       | https://viper.li                                |
| Issue Tracker  | https://github.com/viper-framework/viper/issues |
+----------------+-------------------------------------------------+
+--------------------------+------------------------------------------+
| Configuration            |                                          |
+--------------------------+------------------------------------------+
| Configuration File       | /home/user/work/viper/viper/viper.conf |
| Storage Path             | /tmp/viper                               |
| Current Project Database | Engine(sqlite:////tmp/viper/viper.db)    |
+--------------------------+------------------------------------------+
+----------------+---------------------+--------------------+---------------------+
| Active Modules | System dependencies | Min Python Version | Python requirements |
+----------------+---------------------+--------------------+---------------------+
| apk            |                     | 2.7                |                     |
| xor            |                     | 2.7                |                     |
| ida            |                     | 2.7                |                     |
| rat            |                     | 2.7                |                     |
| clamav         |                     | 2.7                |                     |
| pst            |                     | 2.7                |                     |
| email          |                     | 2.7                |                     |
| pe             |                     | 2.7                |                     |
| editdistance   |                     | 2.7                |                     |
| yara           |                     | 2.7                |                     |
| reports        |                     | 2.7                |                     |
| elf            |                     | 2.7                |                     |
| image          |                     | 2.7                |                     |
| html           |                     | 2.7                |                     |
| cuckoo         |                     | 2.7                |                     |
| pssl           |                     | 2.7                |                     |
| pdns           |                     | 2.7                |                     |
| macho          |                     | 2.7                |                     |
| virustotal     |                     | 2.7                |                     |
| idx            |                     | 2.7                |                     |
| fuzzy          |                     | 2.7                |                     |
| jar            |                     | 2.7                |                     |
| r2             |                     | 2.7                |                     |
| debup          |                     | 2.7                |                     |
| size           |                     | 2.7                |                     |
| lastline       |                     | 2.7                |                     |
| shellcode      |                     | 2.7                |                     |
| strings        |                     | 2.7                |                     |
| triage         |                     | 2.7                |                     |
| swf            |                     | 2.7                |                     |
| office         |                     | 2.7                |                     |
| koodous        |                     | 2.7                |                     |
+----------------+---------------------+--------------------+---------------------+
+----------------+---------------------+--------------------+------------------------------+
| Failed Modules | System dependencies | Min Python Version | Python requirements          |
+----------------+---------------------+--------------------+------------------------------+
| misp           |                     | 2.7                | pymisp,pytaxonomies,requests |
| exif           | exif                | 2.7                | exiftool                     |
+----------------+---------------------+--------------------+------------------------------+

[Rafiot]

Should I merge it, or is it still WIP?

[botherder]

I like it.

[frennkie]

I like it

nice to hear that.. :-)

Should I merge it, or is it still WIP?

I think the "backend" code is pretty much complete.. but I'm not entirely happy with the way the information is presented to the user.. but I haven't come up with a nicer idea.

Additionally I think we should identify all the dependency for the modules before merging into master... at the moment this is more a PoC than a full (documented) PR...

I added the tasks as I see them and will try to find some time the next days to work on this (thanks for the reminder.. ;-) )

[botherder]

I still like this feature, but conflicts need to be resolved.
I also would like to brainstorm about the possibility of not downloading all dependencies for all modules by default as it is right now, but trigger a download only when a module is being used. Any thoughts?

[botherder]

I am still keen on reviewing whether this would be useful. Particularly now that I split the modules away from the core and into a dedicated repository.
If this is obsolete, then please close this PR. Thanks!