Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make CPE matching case insensitive #2

Merged
merged 7 commits into from
Dec 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .envrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
source_url "https://raw.githubusercontent.com/cachix/devenv/82c0147677e510b247d8b9165c54f73d32dfd899/direnvrc" "sha256-7u4iDd1nZpxL4tCzmPG0dQgC5V+/44Ba+tHkPob1v2k="

use devenv
8 changes: 6 additions & 2 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,22 +12,26 @@ jobs:
strategy:
fail-fast: false
matrix:
python-version: ["3.8", "3.9", "3.10", "3.11"]
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12", "3.13"]

steps:
- uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install poetry
poetry install --all-extras

- name: Pre-commit checks
run: |
poetry run pre-commit run -a
poetry run -- pre-commit run --config .pre-commit-ci.yaml --all-files

- name: Test
run: |
poetry run pytest
11 changes: 5 additions & 6 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
*.py[cod]
.venvs/

# C extensions
*.so
Expand Down Expand Up @@ -33,10 +34,8 @@ nosetests.xml
output/*.html
output/*/index.html

# Sphinx
docs/_build
.devenv*
devenv.local.nix

# Cookiecutter
output/
.pytest_cache/
dist/
.direnv
.pre-commit-config.yaml
8 changes: 3 additions & 5 deletions .pre-commit-config.yaml → .pre-commit-ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# See https://pre-commit.com/hooks.html for more hooks
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.4.0
rev: v5.0.0
hooks:
- id: trailing-whitespace
exclude: ".*\\.md"
Expand All @@ -13,13 +13,11 @@ repos:
- id: check-added-large-files

- repo: https://github.com/charliermarsh/ruff-pre-commit
rev: "v0.1.2"
rev: "v0.7.3"
hooks:
- id: ruff
name: Check python (ruff)
args: [--show-source, --fix, --show-fixes, --exit-non-zero-on-fix]
- id: ruff-format
name: Format python (ruff)
args: [--config, pyproject.toml]

- repo: https://github.com/python-jsonschema/check-jsonschema
rev: 0.21.0
Expand Down
24 changes: 24 additions & 0 deletions Justfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
help:
@just --list

# Install all supported Python versions and dependencies in separate virtualenvs
install-all:
#!/usr/bin/env bash
for version in 3.8 3.9 3.10 3.11 3.12 3.13; do \
run-python-version $version poetry install
done

# Run tests with all supported Python versions
test-all:
#!/usr/bin/env bash
for version in 3.8 3.9 3.10 3.11 3.12 3.13; do
run-python-version $version python$version -m pytest
done

# Run checks (lints, formatting)
check:
pre-commit run --all-files

# Completely wipe the development environment and start from scratch
reset-devenv:
rm -rf .venvs .direnv .pre-commit-config.yaml .pytest_cache .ruff_cache
21 changes: 15 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,5 @@
# cpematcher

[![Build Status](https://travis-ci.com/alertot/cpematcher.svg?branch=master)](https://travis-ci.com/alertot/cpematcher) [![PyPIersion](https://badge.fury.io/py/cpematcher.svg)](https://badge.fury.io/py/cpematcher)

## Overview

Match and compare CPEs.

## Installation / Usage
Expand All @@ -12,8 +8,21 @@ To install use pip:

$ pip install cpematcher-ng



# Origin

The project was forked from [cpematcher](https://github.com/alertot/cpematcher). Kudos to the original author!

# Development

We use [devenv](https://devenv.sh) and [direnv]() for development.
You can setup virtualenvs for all supported Python versions:

```shell
$ just install-versions
```

And run all checks and tests after that:

```shell
$ just check test-all
```
195 changes: 95 additions & 100 deletions cpematcher/core.py
Original file line number Diff line number Diff line change
@@ -1,45 +1,21 @@
import fnmatch
from typing import Optional, Tuple

from .utils import split_cpe_string
from .version import Version

OR_OPERATOR = "OR"
AND_OPERATOR = "AND"
CPEv23 = "cpe:2.3:"


class CPE:
cpe23_start = "cpe:2.3:"
fields = [
"part",
"vendor",
"product",
"version",
"update",
"edition",
"language",
"sw_edition",
"target_sw",
"target_hw",
"other",
]

@property
def matches_all(self):
martonilles marked this conversation as resolved.
Show resolved Hide resolved
return self.version == "*" and not (
self.version_start_including
or self.version_start_excluding
or self.version_end_including
or self.version_end_excluding
)

def __init__(
self,
cpe_str,
vulnerable=True,
version_start_including=None,
version_start_excluding=None,
version_end_including=None,
version_end_excluding=None,
cpe_str: str,
vulnerable: bool = True,
version_start_including: Optional[str] = None,
version_start_excluding: Optional[str] = None,
version_end_including: Optional[str] = None,
version_end_excluding: Optional[str] = None,
):
"""Create CPE object with information about affected software.

Expand All @@ -48,78 +24,101 @@ def __init__(
then we added the argument `vulnerable`.

There are some examples in CVE database.

"""
assert cpe_str.startswith(self.cpe23_start), "Only CPE 2.3 is supported"
cpe_str = cpe_str.replace(self.cpe23_start, "")

values = split_cpe_string(cpe_str)
if len(values) != 11:
raise ValueError("Incomplete number of fields")

for f in self.fields:
setattr(self, f, values.pop(0))
assert cpe_str.startswith(CPEv23), "Only CPE 2.3 is supported"

attr_values = split_cpe_string(cpe_str)
if len(attr_values) != 13:
raise ValueError("Incomplete number of CPE attributes")

(
*_,
self.part,
self.vendor,
self.product,
self.version_str,
self.update,
self.edition,
self.language,
self.sw_edition,
self.target_sw,
self.target_hw,
self.other,
) = attr_values

self.is_vulnerable = vulnerable

self.version = Version(self.version_str)
self.version_start_including = Version(version_start_including)
self.version_start_excluding = Version(version_start_excluding)
self.version_end_including = Version(version_end_including)
self.version_end_excluding = Version(version_end_excluding)

def matches(self, another_cpe): # noqa: C901
"""Verify if `another_cpe` matches, first through field comparison and
then using the border constraints.
@property
def no_version(self) -> Tuple[str, str, str, str, str, str, str, str, str, str]:
return (
self.part,
self.vendor,
self.product,
self.update,
self.edition,
self.language,
self.sw_edition,
self.target_sw,
self.target_hw,
self.other,
)

def matches(self, other: "CPE") -> bool:
"""Verify if `other` matches, first through attribute comparison
then using version matching and border constraints.
"""
for f in self.fields:
value = getattr(self, f)
another_value = getattr(another_cpe, f)
"""
Depending on the order, fnmatch.fnmatch could return False
if wildcard is the first value.
As wildcard should always return True in any case,
we reorder the arguments based on that.
"""
if another_value == "*":
order = [value, another_value]
else:
order = [another_value, value]

if (
f == "version"
and "*" not in order
and "*" not in order[0]
and "*" not in order[1]
):
if Version(order[0]) != Version(order[1]):
return False
elif not fnmatch.fnmatch(*order):
return False

version = Version(another_cpe.version)
return self._matches_fields(other) and self._matches_version(other)

@staticmethod
def _glob_equal(value1: str, value2: str) -> bool:
value1, value2 = value1.lower(), value2.lower()
# Depending on the order, fnmatch.fnmatch could return False if wildcard
# is the first value. As wildcard should always return True in any case,
# we reorder the arguments based on that.
glob_values = [value1, value2] if value2 == "*" else [value2, value1]
return fnmatch.fnmatch(*glob_values)

def _matches_fields(self, other: "CPE") -> bool:
return all(
self._glob_equal(value, other_value)
for value, other_value in zip(self.no_version, other.no_version)
)

# Do verifications on start version
if self.version_start_including and version < self.version_start_including:
def _matches_version(self, other: "CPE") -> bool: # noqa: C901
if "*" in self.version_str or "*" in other.version_str:
if not self._glob_equal(self.version_str, other.version_str):
return False
elif self.version != other.version:
return False

if self.version_start_excluding and version <= self.version_start_excluding:
if (
self.version_start_including
and other.version < self.version_start_including
):
return False

if self.version_end_including and version > self.version_end_including:
if (
self.version_start_excluding
and other.version <= self.version_start_excluding
):
return False

if self.version_end_excluding and version >= self.version_end_excluding:
if self.version_end_including and other.version > self.version_end_including:
return False
if self.version_end_excluding and other.version >= self.version_end_excluding:
return False

# ruff: noqa: SIM103
return True


class CPEOperation:
"""Handle operations defined on CPE sets.

Support for:
- OR operations

Support only OR operations.
"""

VERSION_MAP = {
Expand All @@ -139,25 +138,21 @@ def _get_value(self, cpe_dict, key):
def __init__(self, operation_dict):
self.cpes = set()

operator = operation_dict["operator"]
if operation_dict["operator"] != "OR":
return None

if operator == OR_OPERATOR:
for cpe_dict in operation_dict["cpe"]:
c = CPE(
cpe_dict["cpe23Uri"],
cpe_dict.get("vulnerable"),
version_start_including=self._get_value(cpe_dict, "vsi"),
version_start_excluding=self._get_value(cpe_dict, "vse"),
version_end_including=self._get_value(cpe_dict, "vei"),
version_end_excluding=self._get_value(cpe_dict, "vee"),
)
for cpe_dict in operation_dict["cpe"]:
cpe = CPE(
cpe_dict["cpe23Uri"],
cpe_dict.get("vulnerable"),
version_start_including=self._get_value(cpe_dict, "vsi"),
version_start_excluding=self._get_value(cpe_dict, "vse"),
version_end_including=self._get_value(cpe_dict, "vei"),
version_end_excluding=self._get_value(cpe_dict, "vee"),
)

self.cpes.add(c)
self.cpes.add(cpe)

def matches(self, another_cpe):
def matches(self, other: "CPE") -> Optional["CPE"]:
"""Return matching CPE object."""
for cpe in self.cpes:
if cpe.matches(another_cpe):
return cpe

return None
return next((cpe for cpe in self.cpes if cpe.matches(other)), None)
3 changes: 2 additions & 1 deletion cpematcher/utils.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
import contextlib
from typing import List


# heavily inspired by https://stackoverflow.com/a/21882672
def split_cpe_string(string):
def split_cpe_string(string: str) -> List[str]:
ret = []
current = []
itr = iter(string)
Expand Down
Loading
Loading