diff --git a/.VERSION b/.VERSION new file mode 100644 index 0000000..be0bb0a --- /dev/null +++ b/.VERSION @@ -0,0 +1 @@ +$Format:%(describe:tags)$ \ No newline at end of file diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..7c5b460 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,2 @@ +# Save the version details for git tarballs in .VERSION file +.VERSION export-subst diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md new file mode 100644 index 0000000..7631400 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -0,0 +1,26 @@ +--- +name: Bug report +about: Create a report to help us improve +title: 'BUG: ' +labels: bug, design needed, enhancement +assignees: '' + +--- + +**Describe the bug** +A clear and concise description of what the bug is. + +**To Reproduce** +Steps to reproduce the behavior: +1. Go to '...' +2. Click on '....' +3. Scroll down to '....' +4. See error + +**Expected behavior** +A clear and concise description of what you expected to happen. + +**Screenshots** +If applicable, add screenshots to help explain your problem. + +**Context (OS, Browser, Device, etc.):** diff --git a/.github/ISSUE_TEMPLATE/dejacode-documentation.md b/.github/ISSUE_TEMPLATE/dejacode-documentation.md new file mode 100644 index 0000000..7e169cf --- /dev/null +++ b/.github/ISSUE_TEMPLATE/dejacode-documentation.md @@ -0,0 +1,14 @@ +--- +name: DejaCode documentation +about: Request a DejaCode documentation improvement +title: 'DOC: ' +labels: documentation +assignees: '' + +--- + +**What type of documentation would you like?** +How-to, Reference, Tutorial, on-screen prompt + +**Documentation topic** +Describe the features of DejaCode that would benefit from more explanation. diff --git a/.github/ISSUE_TEMPLATE/dejacode-enhancement-request.md b/.github/ISSUE_TEMPLATE/dejacode-enhancement-request.md new file mode 100644 index 0000000..93f2d33 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/dejacode-enhancement-request.md @@ -0,0 +1,20 @@ +--- +name: DejaCode enhancement request +about: Suggest an enhancement for DejaCode +title: 'Enhancement request: ' +labels: design needed, enhancement +assignees: '' + +--- + +**Is your enhancement request related to a problem? Please describe.** +Describe the problem that you would like to address. + +**What are the benefits of the requested enhancement?** +How will the new functionality benefit DejaCode users? + +**Describe the solution you would like** +Provide a concise scenario or use case that needs to be supported in DejaCode. + +**Additional notes** +Add any other context or screenshots about the feature request here. diff --git a/.github/ISSUE_TEMPLATE/roadmap-item-template.md b/.github/ISSUE_TEMPLATE/roadmap-item-template.md new file mode 100644 index 0000000..aacc4dc --- /dev/null +++ b/.github/ISSUE_TEMPLATE/roadmap-item-template.md @@ -0,0 +1,17 @@ +--- +name: Roadmap item template +about: Structure for roadmap items +title: 'RFC: ' +labels: design needed, enhancement +assignees: '' + +--- + +**Summary** +A clear and concise description of the Roadmap requirements and objectives. + +**Intended Outcome** +A clear and concise description of the impact on the AboutCode stack. + +**How will it work?** +Details to explain what needs to be done. diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 31f65a4..3dbb5e3 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -33,9 +33,9 @@ jobs: uses: actions/checkout@v4 - name: Set up Python - uses: actions/setup-python@v4 + uses: actions/setup-python@v5 with: - python-version: "3.10" + python-version: "3.12" - name: Install python-ldap OS dependencies run: sudo apt-get install -y libsasl2-dev libldap2-dev libssl-dev diff --git a/.github/workflows/gh-release.yml b/.github/workflows/gh-release.yml new file mode 100644 index 0000000..c23b238 --- /dev/null +++ b/.github/workflows/gh-release.yml @@ -0,0 +1,17 @@ +name: Create a GitHub release + +on: + workflow_dispatch: + push: + tags: + - "v*.*.*" + +jobs: + create-github-release: + runs-on: ubuntu-22.04 + + steps: + - name: Create a GitHub release + uses: softprops/action-gh-release@v1 + with: + draft: false diff --git a/.gitignore b/.gitignore index 23cd2bd..e2a18e1 100644 --- a/.gitignore +++ b/.gitignore @@ -1,5 +1,6 @@ *.pyc *.db +*.rdb .installed.cfg parts develop-eggs diff --git a/.readthedocs.yaml b/.readthedocs.yaml index 5399ec0..19c4747 100644 --- a/.readthedocs.yaml +++ b/.readthedocs.yaml @@ -8,7 +8,7 @@ version: 2 build: os: ubuntu-22.04 tools: - python: "3.10" + python: "3.12" # Optionally declare the Python requirements required to build your docs python: diff --git a/CHANGELOG.rst b/CHANGELOG.rst index 053b324..173519d 100644 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -1,11 +1,101 @@ Release notes ============= -### Version 5.0.1-dev +### Version 5.1.1-dev + +- Add visual indicator in hierarchy views, when an object on the far left or far right + also belong or have a hierarchy (relathionship tree). + https://github.com/nexB/dejacode/issues/70 + +### Version 5.1.0 + +- Upgrade Python version to 3.12 and Django to 5.0.x + https://github.com/nexB/dejacode/issues/50 + +- Replace Celery by RQ for async job queue and worker. + https://github.com/nexB/dejacode/issues/6 + +- Add support for CycloneDX spec version "1.6". + In the UI and API, older spe version such as "1.4" and "1.5" are also available as + download. + https://github.com/nexB/dejacode/pull/79 + +- Lookup in PurlDB by purl in Add Package form. + When a Package URL is available in the context of the "Add Package" form, + for example when using a link from the Vulnerabilities tab, + data is fetched from the PurlDB to initialize the form. + https://github.com/nexB/dejacode/issues/47 + +- If you select two versions of the same Product in the Product list, or two different + Products, and click the Compare button, you can now download the results of the + comparison to a .xlsx file, making it easy to share the information with your + colleagues. + https://github.com/nexB/dejacode/issues/7 + +- Add dark theme support in UI. + https://github.com/nexB/dejacode/issues/25 + +- Add "Load Packages from SBOMs", "Import scan results", and + "Pull ScanCode.io project data" feature as Product action in the REST API. + https://github.com/nexB/dejacode/issues/59 + +- Add REST API endpoints to download SBOMs as CycloneDX and SPDX. + https://github.com/nexB/dejacode/issues/60 + +- Refactor the "Import manifest" feature as "Load SBOMs". + https://github.com/nexB/dejacode/issues/61 + +- Add support to import packages from manifest. + https://github.com/nexB/dejacode/issues/65 + +- Add a vulnerability link to the VulnerableCode app in the Vulnerability tab. + https://github.com/nexB/dejacode/issues/4 + +- Add a DEJACODE_SUPPORT_EMAIL setting for support email address customization. + https://github.com/nexB/dejacode/issues/76 + +- Show the individual PURL fields in the Package details view. + https://github.com/nexB/dejacode/issues/83 + +- Fix the logout link of the admin app. + https://github.com/nexB/dejacode/issues/89 + +- Display full commit in the version displayed in the UI + https://github.com/nexB/dejacode/issues/88 + +- Refine the Product comparison logic for Packages. + The type and namespace fields are now used along the name field to match similar + Packages (excluding the version). + https://github.com/nexB/dejacode/issues/113 + +- Refactor the implementation of Keywords on forms to allow more flexibilty. + Existing Keywords are suggested for consistency but any values is now allowed. + https://github.com/nexB/dejacode/issues/48 + +- Display Product inventory count on the Product list view. + https://github.com/nexB/dejacode/issues/81 + +- Always display the full Package URL in the UI view including the "pkg:" prefix. + https://github.com/nexB/dejacode/issues/115 + +- Add a new AboutCode tab in Package details view. + https://github.com/nexB/dejacode/issues/42 + +- Enhance Package Import to support modifications. + https://github.com/nexB/dejacode/issues/84 + +- Add an option on the "Add to Product" form to to replace any existing relationships + with a different version of the same object by the selected object. + https://github.com/nexB/dejacode/issues/12 + +### Version 5.0.1 - Improve the stability of the "Check for new Package versions" feature. https://github.com/nexB/dejacode/issues/17 +- Improve the support for SourgeForge download URLs. + https://github.com/nexB/dejacode/issues/26 + ### Version 5.0.0 Initial release. diff --git a/Dockerfile b/Dockerfile index b550c72..1cad2a3 100644 --- a/Dockerfile +++ b/Dockerfile @@ -6,7 +6,7 @@ # See https://aboutcode.org for more information about AboutCode FOSS projects. # -FROM python:3.10-slim +FROM python:3.12-slim LABEL org.opencontainers.image.source="https://github.com/nexB/dejacode" LABEL org.opencontainers.image.description="DejaCode" @@ -31,6 +31,7 @@ RUN apt-get update \ libldap2-dev \ libsasl2-dev \ libpq5 \ + git \ wait-for-it \ && apt-get clean \ && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* diff --git a/Makefile b/Makefile index 7e69bde..f914f48 100644 --- a/Makefile +++ b/Makefile @@ -6,7 +6,7 @@ # See https://aboutcode.org for more information about AboutCode FOSS projects. # -PYTHON_EXE=python3.10 +PYTHON_EXE=python3.12 MANAGE=bin/python manage.py ACTIVATE?=. bin/activate; PIP_ARGS=--find-links=./thirdparty/dist/ --no-index --no-cache-dir @@ -129,7 +129,10 @@ postgresdb: @gunzip < ${DB_INIT_FILE} | psql --username=${DB_USERNAME} ${DB_NAME} run: - ${MANAGE} runserver 8000 + ${MANAGE} runserver 8000 --insecure + +worker: + ${MANAGE} rqworker test: @echo "-> Run the test suite" @@ -162,4 +165,4 @@ log: createsuperuser: ${DOCKER_EXEC} web ./manage.py createsuperuser -.PHONY: virtualenv conf dev envfile check bandit isort black doc8 valid check-docstrings check-deploy clean initdb postgresdb migrate run test docs build psql bash shell log createsuperuse +.PHONY: virtualenv conf dev envfile check bandit isort black doc8 valid check-docstrings check-deploy clean initdb postgresdb migrate run test docs build psql bash shell log createsuperuser diff --git a/component_catalog/api.py b/component_catalog/api.py index 33c2bd9..e9ad4fb 100644 --- a/component_catalog/api.py +++ b/component_catalog/api.py @@ -35,13 +35,16 @@ from dejacode_toolkit.download import collect_package_data from dejacode_toolkit.scancodeio import ScanCodeIO from dje import tasks +from dje.api import AboutCodeFilesActionMixin from dje.api import CreateRetrieveUpdateListViewSet +from dje.api import CycloneDXSOMActionMixin from dje.api import DataspacedAPIFilterSet from dje.api import DataspacedHyperlinkedRelatedField from dje.api import DataspacedSerializer from dje.api import DataspacedSlugRelatedField from dje.api import ExternalReferenceSerializer from dje.api import NameVersionHyperlinkedRelatedField +from dje.api import SPDXDocumentActionMixin from dje.filters import LastModifiedDateFilter from dje.filters import MultipleCharFilter from dje.filters import MultipleUUIDFilter @@ -447,7 +450,9 @@ class Meta: ) -class ComponentViewSet(CreateRetrieveUpdateListViewSet): +class ComponentViewSet( + SPDXDocumentActionMixin, CycloneDXSOMActionMixin, CreateRetrieveUpdateListViewSet +): queryset = Component.objects.all() serializer_class = ComponentSerializer filterset_class = ComponentFilterSet @@ -820,7 +825,13 @@ def collect_create_scan(download_url, user): return package -class PackageViewSet(SendAboutFilesMixin, CreateRetrieveUpdateListViewSet): +class PackageViewSet( + SendAboutFilesMixin, + AboutCodeFilesActionMixin, + SPDXDocumentActionMixin, + CycloneDXSOMActionMixin, + CreateRetrieveUpdateListViewSet, +): queryset = Package.objects.all() serializer_class = PackageSerializer filterset_class = PackageAPIFilterSet @@ -868,13 +879,6 @@ def about(self, request, uuid): package = self.get_object() return Response({"about_data": package.as_about_yaml()}) - @action(detail=True) - def about_files(self, request, uuid): - package = self.get_object() - about_files = package.get_about_files() - filename = self.get_filename(package) - return self.get_zipped_response(about_files, filename) - download_url_description = ( "A single, or list of, Download URL(s).

" 'cURL style: -d "download_url=url1&download_url=url2"

' @@ -1025,3 +1029,37 @@ def get_queryset(self): "child", ) ) + + +class KeywordSerializer(DataspacedSerializer): + class Meta: + model = ComponentKeyword + fields = ( + "api_url", + "uuid", + "label", + "description", + ) + extra_kwargs = { + "api_url": { + "view_name": "api_v2:componentkeyword-detail", + "lookup_field": "uuid", + }, + } + + +class KeywordViewSet(CreateRetrieveUpdateListViewSet): + queryset = ComponentKeyword.objects.all() + serializer_class = KeywordSerializer + lookup_field = "uuid" + search_fields = ( + "label", + "description", + ) + search_fields_autocomplete = ("label",) + ordering_fields = ( + "label", + "created_date", + "last_modified_date", + ) + allow_reference_access = True diff --git a/component_catalog/filters.py b/component_catalog/filters.py index f8eb067..353df10 100644 --- a/component_catalog/filters.py +++ b/component_catalog/filters.py @@ -71,7 +71,7 @@ class ComponentFilterSet(DataspacedFilterSet): label=_("License"), field_name="licenses__key", to_field_name="key", - queryset=License.objects.all().only("key", "short_name", "dataspace"), + queryset=License.objects.only("key", "short_name", "dataspace__id"), widget=BootstrapSelectMultipleWidget( search_placeholder="Search licenses", ), @@ -80,7 +80,7 @@ class ComponentFilterSet(DataspacedFilterSet): label=_("Keyword"), to_field_name="label", lookup_expr="contains", - queryset=ComponentKeyword.objects.all().only("label", "dataspace"), + queryset=ComponentKeyword.objects.only("label", "dataspace__id"), widget=BootstrapSelectMultipleWidget( search_placeholder="Search keywords", ), @@ -183,7 +183,7 @@ class PackageFilterSet(DataspacedFilterSet): label=_("License"), field_name="licenses__key", to_field_name="key", - queryset=License.objects.all().only("key", "short_name", "dataspace"), + queryset=License.objects.only("key", "short_name", "dataspace__id"), widget=BootstrapSelectMultipleWidget( search_placeholder="Search licenses", ), diff --git a/component_catalog/forms.py b/component_catalog/forms.py index f5304fc..e6c3ddf 100644 --- a/component_catalog/forms.py +++ b/component_catalog/forms.py @@ -39,7 +39,7 @@ from dje.forms import DataspacedModelForm from dje.forms import DefaultOnAdditionLabelMixin from dje.forms import Group -from dje.forms import JSONListChoiceField +from dje.forms import JSONListField from dje.forms import OwnerChoiceField from dje.forms import autocomplete_placeholder from dje.mass_update import DejacodeMassUpdateForm @@ -56,23 +56,29 @@ class SetKeywordsChoicesFormMixin: def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) - - keywords_field = self.fields.get("keywords") - if keywords_field: + if keywords_field := self.fields.get("keywords"): keywords_qs = ComponentKeyword.objects.scope(self.dataspace) labels = keywords_qs.values_list("label", flat=True) - keywords_field.choices = [(label, label) for label in labels] - keywords_field.widget.attrs.update( - { - "data-list": ", ".join(labels), - } - ) + keywords_field.widget.attrs.update({"data-list": ", ".join(labels)}) + + +class KeywordsField(JSONListField): + def __init__(self, **kwargs): + widget = AutocompleteInput( + attrs={ + "data-api_url": reverse_lazy("api_v2:componentkeyword-list"), + }, + display_link=False, + display_attribute="label", + ) + kwargs.setdefault("widget", widget) + kwargs.setdefault("required", False) + super().__init__(**kwargs) class ComponentForm( LicenseExpressionFormMixin, DefaultOnAdditionLabelMixin, - SetKeywordsChoicesFormMixin, DataspacedModelForm, ): default_on_addition_fields = ["configuration_status"] @@ -83,10 +89,7 @@ class ComponentForm( ] color_initial = True - keywords = JSONListChoiceField( - required=False, - widget=AwesompleteInputWidget(attrs=autocomplete_placeholder), - ) + keywords = KeywordsField() packages_ids = forms.CharField( widget=forms.HiddenInput, @@ -262,16 +265,12 @@ def clean(self): class PackageForm( LicenseExpressionFormMixin, PackageFieldsValidationMixin, - SetKeywordsChoicesFormMixin, DataspacedModelForm, ): save_as = True color_initial = True - keywords = JSONListChoiceField( - required=False, - widget=AwesompleteInputWidget(attrs=autocomplete_placeholder), - ) + keywords = KeywordsField() collect_data = forms.BooleanField( required=False, @@ -647,6 +646,17 @@ class AddToProductAdminForm(forms.Form): queryset=Product.objects.none(), ) ids = forms.CharField(widget=forms.widgets.HiddenInput) + replace_existing_version = forms.BooleanField( + required=False, + initial=False, + label="Replace existing relationships by newer version.", + help_text=( + "Select this option to replace any existing relationships with a different version " + "of the same object. " + "If more than one version of the object is already assigned, no replacements will be " + "made, and the new version will be added instead." + ), + ) def __init__(self, request, model, relation_model, *args, **kwargs): super().__init__(*args, **kwargs) @@ -664,9 +674,11 @@ def get_selected_objects(self): def save(self): product = self.cleaned_data["product"] + return product.assign_objects( related_objects=self.get_selected_objects(), user=self.request.user, + replace_version=self.cleaned_data["replace_existing_version"], ) @@ -720,15 +732,15 @@ def new_component_from_package_link(self): href = f"{component_add_url}?package_ids={package.id}" return HTML( + f"
" f'
' f' ' f" Add Component from Package data" f" " f"
" - f"
" ) def clean_component(self): @@ -750,9 +762,9 @@ def helper(self): helper.layout = Layout( Fieldset( None, - self.new_component_from_package_link(), "object_id", "component", + self.new_component_from_package_link(), ), ) return helper @@ -910,7 +922,7 @@ class ComponentAdminForm( SetKeywordsChoicesFormMixin, DataspacedAdminForm, ): - keywords = JSONListChoiceField( + keywords = JSONListField( required=False, widget=AdminAwesompleteInputWidget(attrs=autocomplete_placeholder), ) @@ -944,7 +956,7 @@ class PackageAdminForm( SetKeywordsChoicesFormMixin, DataspacedAdminForm, ): - keywords = JSONListChoiceField( + keywords = JSONListField( required=False, widget=AdminAwesompleteInputWidget(attrs=autocomplete_placeholder), ) @@ -990,7 +1002,7 @@ class ComponentMassUpdateForm( DejacodeMassUpdateForm, ): raw_id_fields = ["owner"] - keywords = JSONListChoiceField( + keywords = JSONListField( required=False, widget=AwesompleteInputWidget(attrs=autocomplete_placeholder), ) @@ -1063,7 +1075,7 @@ class Meta: class PackageMassUpdateForm( LicenseExpressionFormMixin, SetKeywordsChoicesFormMixin, DejacodeMassUpdateForm ): - keywords = JSONListChoiceField( + keywords = JSONListField( required=False, widget=AwesompleteInputWidget(attrs=autocomplete_placeholder), ) diff --git a/component_catalog/importers.py b/component_catalog/importers.py index 70153e7..f60467e 100644 --- a/component_catalog/importers.py +++ b/component_catalog/importers.py @@ -5,6 +5,7 @@ # See https://github.com/nexB/dejacode for support or download. # See https://aboutcode.org for more information about AboutCode FOSS projects. # + import json import os from urllib.parse import urlparse @@ -29,7 +30,8 @@ from component_catalog.models import Package from component_catalog.models import Subcomponent from component_catalog.programming_languages import PROGRAMMING_LANGUAGES -from dje.forms import JSONListChoiceField +from dje.fields import SmartFileField +from dje.forms import JSONListField from dje.importers import BaseImporter from dje.importers import BaseImportModelForm from dje.importers import ComponentRelatedFieldImportMixin @@ -40,6 +42,12 @@ from product_portfolio.models import ProductComponent from product_portfolio.models import ProductPackage +keywords_help = ( + get_help_text(Component, "keywords") + + " You can add multiple keywords by separating them with commas, like this: " + "keyword1, keyword2." +) + class OwnerChoiceField(ModelChoiceFieldForImport): def get_suggestions(self, value, limit=5): @@ -157,8 +165,9 @@ class ComponentImportForm( identifier_field="label", ) - keywords = JSONListChoiceField( + keywords = JSONListField( required=False, + help_text=keywords_help, ) acceptable_linkages = ImportMultipleChoiceField( @@ -172,6 +181,7 @@ class Meta: "children", "packages", "completion_level", + "request_count", # JSONField not supported "dependencies", ) @@ -237,8 +247,9 @@ class PackageImportForm( identifier_field="label", ) - keywords = JSONListChoiceField( + keywords = JSONListField( required=False, + help_text=keywords_help, ) class Meta: @@ -248,6 +259,7 @@ class Meta: # JSONField not supported "dependencies", "file_references", + "request_count", "parties", ] @@ -294,10 +306,20 @@ def save(self, commit=True): return package +class PackageImportableUploadFileForm(forms.Form): + file = SmartFileField(extensions=["csv", "json"]) + + @property + def header(self): + return "Select a CSV (.csv) or JSON (.json) file" + + class PackageImporter(BaseImporter): model_form = PackageImportForm + upload_form_class = PackageImportableUploadFileForm add_to_product_perm = "product_portfolio.add_productpackage" relation_model = ProductPackage + update_existing = True def prepare_data_json(self, data): """ diff --git a/component_catalog/models.py b/component_catalog/models.py index 9ae9e40..e4917f4 100644 --- a/component_catalog/models.py +++ b/component_catalog/models.py @@ -34,6 +34,8 @@ from attributecode.model import About from cyclonedx import model as cyclonedx_model from cyclonedx.model import component as cyclonedx_component +from cyclonedx.model import contact as cyclonedx_contact +from cyclonedx.model import license as cyclonedx_license from packageurl import PackageURL from packageurl.contrib import purl2url from packageurl.contrib import url2purl @@ -62,7 +64,7 @@ from dje.models import ParentChildModelMixin from dje.models import ParentChildRelationshipModel from dje.models import ReferenceNotesMixin -from dje.tasks import tasks_logger +from dje.tasks import logger as tasks_logger from dje.utils import set_fields_from_object from dje.validators import generic_uri_validator from dje.validators import validate_url_segment @@ -758,7 +760,7 @@ def as_cyclonedx(self, license_expression_spdx=None): """Return this Component/Product as an CycloneDX Component entry.""" supplier = None if self.owner: - supplier = cyclonedx_model.OrganizationalEntity( + supplier = cyclonedx_contact.OrganizationalEntity( name=self.owner.name, urls=[self.owner.homepage_url], ) @@ -766,9 +768,9 @@ def as_cyclonedx(self, license_expression_spdx=None): expression_spdx = license_expression_spdx or self.get_license_expression_spdx_id() licenses = [] if expression_spdx: - licenses = [ - cyclonedx_model.LicenseChoice(license_expression=expression_spdx), - ] + # Using the LicenseExpression directly as the make_with_expression method + # does not support the "LicenseRef-" keys. + licenses = [cyclonedx_license.LicenseExpression(value=expression_spdx)] if self.__class__.__name__ == "Product": component_type = cyclonedx_component.ComponentType.APPLICATION @@ -777,12 +779,12 @@ def as_cyclonedx(self, license_expression_spdx=None): return cyclonedx_component.Component( name=self.name, - component_type=component_type, + type=component_type, version=self.version, bom_ref=str(self.uuid), supplier=supplier, licenses=licenses, - copyright_=self.copyright, + copyright=self.copyright, description=self.description, cpe=getattr(self, "cpe", None), properties=get_cyclonedx_properties(self), @@ -1839,10 +1841,9 @@ def enforce_identifier(self): def identifier(self): """ Provide a unique value to identify each Package. - It is the Package URL (minus the 'pkg:' prefix) if one exists; - otherwise it is the Package Filename. + It is the Package URL if one exists; otherwise it is the Package Filename. """ - return self.short_package_url or self.filename + return self.package_url or self.filename @classmethod def identifier_help(cls): @@ -1873,8 +1874,8 @@ def package_url_filename(self): Return the Package URL string as a valid filename. Useful when `Package.filename` is not available. """ - cleaned_package_url = self.short_package_url - for char in "/@?=#": + cleaned_package_url = self.plain_package_url + for char in "/@?=#:": cleaned_package_url = cleaned_package_url.replace(char, "_") return get_valid_filename(cleaned_package_url) @@ -1887,7 +1888,8 @@ def get_url(self, name, params=None, include_identifier=False): if not params: params = [self.dataspace.name, quote_plus(str(self.uuid))] if include_identifier: - params.insert(1, self.identifier) + # For the URL, using plain_package_url for simplification + params.insert(1, self.plain_package_url or self.filename) return super().get_url(name, params) def get_absolute_url(self): @@ -1913,12 +1915,16 @@ def get_export_cyclonedx_url(self): return self.get_url("export_cyclonedx") @classmethod - def get_identifier_fields(cls): + def get_identifier_fields(cls, *args, purl_fields_only=False, **kwargs): """ Explicit list of identifier fields as we do not enforce a unique together on this model. This is used in the Importer, to catch duplicate entries. + The purl_fields_only option can be use to limit the results. """ + if purl_fields_only: + return PACKAGE_URL_FIELDS + return ["filename", "download_url", *PACKAGE_URL_FIELDS] @property @@ -2015,9 +2021,15 @@ def size_formatted(self): @cached_property def component(self): - """Return the Component instance if 1 and only 1 Component is assigned to this Package.""" - with suppress(ObjectDoesNotExist, MultipleObjectsReturned): - return self.component_set.get() + """ + Return the Component instance if 1 and only 1 Component is assigned to this + Package. + Using ``component_set.all()`` to benefit from prefetch_related when it was + applied to the Package QuerySet. + """ + component_set = self.component_set.all() + if len(component_set) == 1: + return component_set[0] def set_values_from_component(self, component, user): changed_fields = set_fields_from_object( @@ -2222,9 +2234,9 @@ def as_cyclonedx(self, license_expression_spdx=None): licenses = [] if expression_spdx: - licenses = [ - cyclonedx_model.LicenseChoice(license_expression=expression_spdx), - ] + # Using the LicenseExpression directly as the make_with_expression method + # does not support the "LicenseRef-" keys. + licenses = [cyclonedx_license.LicenseExpression(value=expression_spdx)] hash_fields = { "md5": cyclonedx_model.HashAlgorithm.MD5, @@ -2233,19 +2245,19 @@ def as_cyclonedx(self, license_expression_spdx=None): "sha512": cyclonedx_model.HashAlgorithm.SHA_512, } hashes = [ - cyclonedx_model.HashType(algorithm=algorithm, hash_value=hash_value) + cyclonedx_model.HashType(alg=algorithm, content=hash_value) for field_name, algorithm in hash_fields.items() if (hash_value := getattr(self, field_name)) ] - purl = self.package_url + package_url = self.get_package_url() return cyclonedx_component.Component( name=self.name, version=self.version, - bom_ref=purl or str(self.uuid), - purl=purl, + bom_ref=str(package_url) or str(self.uuid), + purl=package_url, licenses=licenses, - copyright_=self.copyright, + copyright=self.copyright, description=self.description, cpe=self.cpe, author=self.author, diff --git a/component_catalog/templates/admin/component_catalog/add_to_product.html b/component_catalog/templates/admin/component_catalog/add_to_product.html index 854add4..a162143 100644 --- a/component_catalog/templates/admin/component_catalog/add_to_product.html +++ b/component_catalog/templates/admin/component_catalog/add_to_product.html @@ -19,17 +19,22 @@

Add {% trans opts.verbose_name %} to a product

The Product provides you with a group of {% trans opts.verbose_name_plural|title %} that are used together, so that you can generate Attribution documentation for all of the {% trans opts.verbose_name_plural|title %} in that Product.

-

+

{{ form.non_field_errors }} {{ form.product.errors }} {{ form.product }} +
+ {{ form.replace_existing_version }} + {{ form.replace_existing_version.label }} +

{{ form.replace_existing_version.help_text }}

+
{% if perms.product_portfolio.add_product %} {% endif %} {{ form.ct }} {{ form.ids }} -

+

diff --git a/component_catalog/templates/component_catalog/base_component_package_details.html b/component_catalog/templates/component_catalog/base_component_package_details.html index 1cbc1e8..8cc12c8 100644 --- a/component_catalog/templates/component_catalog/base_component_package_details.html +++ b/component_catalog/templates/component_catalog/base_component_package_details.html @@ -20,7 +20,7 @@ {% block javascripts %} {{ block.super }} - + {% include 'includes/dependencies-json-viewer.js.html' %} {% if open_add_to_package_modal %} diff --git a/component_catalog/templates/component_catalog/base_component_package_list.html b/component_catalog/templates/component_catalog/base_component_package_list.html index d90a75c..8d1617d 100644 --- a/component_catalog/templates/component_catalog/base_component_package_list.html +++ b/component_catalog/templates/component_catalog/base_component_package_list.html @@ -2,9 +2,11 @@ {% load i18n %} {% block top-right-buttons %} - + + + {% if form or add_to_component_form %}
{% csrf_token %} -