PulpImport uses incorrect id and overwrites unrelated lists' links #2628

Versions

• pulpcore 3.85.26, pulp_container 2.27.10, django-import-export 4.4.1 (observed)
• pulp_container main (2.31.0.dev, commit 9769f56): the same code is present, see below
• Installed from the upstream pulp/pulp container image; PostgreSQL; Pulp Export/Import (PulpExporter / PulpImporter) between two separate Pulp instances

Summary

ManifestListManifestResource (the import/export resource for the link between a manifest list and each manifest it lists) declares no import_id_fields. django-import-export therefore uses its default, ["id"], and the export writes the local integer primary key of every ManifestListManifest row. On import, a row is looked up by that id and updated in place. The id is only meaningful on the exporting instance.

Two failures follow on the importing instance:

1. Silent overwrite. If the importing instance already has ManifestListManifest rows with the same ids but for a different manifest list, those rows are rewritten to point at the imported list and its children. The unrelated manifest list loses its listed manifests. The import task group completes successfully, and the imported repository looks complete.
2. Hard failure. If the importing instance already has the same (manifest list, listed manifest) pair under a different id, the import tries to insert a second row for the pair and fails with duplicate key value violates unique constraint on (image_manifest_id, manifest_list_id). The task fails and no repository version is created.

Reproduction

Two fresh instances, A (exporting) and B (importing). Manifest lists are used because they are the only content with these link rows.

Overwrite case:

1. On B, sync or push a manifest list P with two listed manifests. Its link rows get ids 1 and 2.
2. On A, sync or push a different manifest list X with two listed manifests. Its link rows also get ids 1 and 2.
3. Export the repository holding X from A with PulpExporter and import it on B with PulpImporter (any repo mapping).
4. On B, read manifest list P: listed_manifests is now empty. X has its two listed manifests, and B has two link rows in total instead of four.

Failure case:

1. On B, create some other link rows first, then sync or push the same manifest list X that A exports. Its pair rows get ids that differ from A's (for example 5 and 6 versus 1 and 2).
2. Export X from A and import it on B.
3. The import task fails with the unique-constraint error above.

The second case also happens on a single instance. Export a repository holding a manifest list, remove its content version and run orphan cleanup (the link rows are deleted), sync the same content again (the pair gets new ids), then import the earlier export.

Every export, full or incremental, includes the link rows of the exported repository version, so both full and incremental imports are affected.

Root cause

pulp_container/app/modelresource.py on main (commit 9769f56), lines 144-178:

class ManifestListManifestResource(QueryModelResource):
    ...
    class Meta:
        model = ManifestListManifest          # line 178: no import_id_fields, no exclude

The other resources in the file set import_id_fields = model.natural_key_fields() (BlobResource, ManifestResource, ManifestSignatureResource, TagResource). This one does not.

With django-import-export 4.4.1, the default import_id_fields = ["id"] (options.py) is used by ModelInstanceLoader.get_instance (instance_loaders.py) to run ManifestListManifest.objects.get(id=<exported id>). If a row exists it is updated, otherwise a row with the exported id is inserted. The model's real identity is unique_together = ("image_manifest", "manifest_list") (models.py:358), and both columns are already exported as digests through ManifestDomainForeignKeyWidget.

No sequence problem occurs: django-import-export resets the sequence after inserts with explicit ids.

Proposed fix

Match rows by the two digest columns and stop exporting the id:

    class Meta:
        model = ManifestListManifest
        exclude = ("id",)
        import_id_fields = ("manifest_list", "image_manifest")

• A row whose (list, child) pair exists is updated in place (platform fields), with no duplicate.
• Any other row is inserted with a new local id.
• Exports written by older versions still carry an id column. It is no longer an import field and is ignored, so older exports import correctly.
• Compatibility note: an export written after the fix has no id column. An unfixed importer reads a missing id as a new row, so it imports it, but it still fails with the unique-constraint error when the same pair exists locally. It does not corrupt other rows.

I verified this change on pulpcore 3.85.26 / pulp_container 2.27.10 with the file patched in place: the unrelated list kept its links, the same pair under a different id was reused, and full and incremental imports of manifest lists, tags, blobs and signatures matched the source by digest.

A PR with the change, a unit test and a functional test follows.

Disclosure: this report and its analysis were drafted with AI assistance (Claude Code) and reviewed by me before filing, per the Pulp AI policy.