docs: add ADR for non-nullable tag external_id

docs/openedx_tagging/decisions/0012-non-nullable-tag-external-id.rst

@@ -0,0 +1,101 @@
+.. _openedx-tagging-adr-0012:
+
+12. Non-nullable tag ``external_id``
+=====================================
+
+Status
+------
+
+Proposed
+
+Context
+-------
+
+`ADR 0010 <0010-mutable-tag-external-id.rst>`_ made ``Tag.external_id`` mutable through an
+import-only rename pathway, to serve as the institution-editable "Competency ID" for
+Competency-Based Education (CBE) tags. That ADR left ``external_id`` nullable, scoped to
+competency taxonomies.
+
+Building the CBE competency workflow surfaced a stronger requirement: every tag needs a
+value for ``external_id`` in order to be able to associate it with objects as part of 
+competency criteria. It's also anticipated that non-CBE taxonomies will want a mandatory,
+institution-assigned identifier in the future, particularly when there becomes a desire to
+convert regular taxonomies into Competency Taxonomies.
+
+Today, ``external_id`` is nullable specifically to work around its own
+``unique_together(taxonomy, external_id)`` constraint: NULL is the only value Postgres,
+MySQL, and SQLite all treat as exempt from a uniqueness constraint, so it's the
+mechanism that lets multiple tags in the same taxonomy have "no external_id" at once.
+Making the field mandatory means every existing NULL row in every current Open edX
+instance needs a real value before the constraint can be added, since this is a
+published library whose migrations run unmodified against arbitrary downstream data.
+
+Decision
+--------
+
+Make ``Tag.external_id`` ``NOT NULL`` at the database level, for every taxonomy, in one
+migration.
+
+- **Backfill for existing rows.** A ``RunPython`` step generates a value for every tag
+  whose ``external_id`` is currently NULL, following the same
+  backfill-then-constrain shape as the ``depth``/``lineage`` migration
+  (``0020_tag_depth_and_lineage.py``): populate real values first, then add the
+  constraint in the same migration once every row satisfies it.
+- **Backfill algorithm.** Take the first ~7 characters of the tag's ``value`` (matching
+  ``external_id``'s existing case-insensitive collation). If that prefix collides with
+  another tag's ``external_id`` already in the same taxonomy, append a numeric counter
+  to the prefix and keep incrementing it until it's unique. This is deterministic and
+  bounded: it never needs an unbounded retry loop, since appending an incrementing
+  counter to an already-short prefix is guaranteed to terminate within the taxonomy's
+  tag count.
+- **New tags going forward.** The REST API and import format keep treating
+  ``external_id`` as optional for the caller, unchanged from today. When a tag is
+  created without one, the same prefix-plus-collision-fallback algorithm generates one
+  automatically, rather than rejecting the request. No existing integration that omits
+  ``external_id`` today has to change.
+- **Institutions can replace an auto-generated value.** If an institution doesn't want
+  the auto-generated ``external_id``, they can change it to their own value through
+  ADR 0010's rename pathway.
+
+This is not treated as a breaking change to the public REST API, and doesn't go through
+the DEPR process. The request side (``external_id`` on tag creation) stays optional, so
+no existing caller has to change what it sends. The response side narrows from
+"nullable string" to "always a string," which is the safe direction for consumers:
+code written to expect a possible ``null`` simply never takes that branch anymore.
+
+Future considerations
+~~~~~~~~~~~~~~~~~~~~~~
+
+If an institution converts an existing regular taxonomy to a CBE taxonomy, its tags
+will already have a non-null ``external_id`` (auto-generated or backfilled) from this
+decision, with no separate migration needed for that conversion.
+
+Rejected Alternatives
+----------------------
+
+Derive backfilled values from the tag's primary key
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Still needs a collision fallback: a tag could already have a hand-assigned
+``external_id`` that happens to match another tag's ``id``. Also rejected because
+``id`` is meant to stay internal (per the README's model conventions, ``uuid`` is the
+documented identifier for external reference), and because it produces a value with no
+relationship to the tag (e.g. ``482913``), which reads as meaningless to institutions.
+
+Require callers to supply ``external_id`` on tag creation
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Simpler mechanically than auto-generation, but rejected because it's a breaking change
+for the REST API and any import/UI flow that omits ``external_id`` today.
+
+Scope this decision to CBE/competency taxonomies only, for now
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Rejected because taxonomies later converted to Competency Taxonomies will want this too.
+
+Changelog
+---------
+
+2026-07-10:
+
+* Proposed.