Merge branch 'main' into jesperhodge/feat--640-cbe-app-foundation

Author: jesperhodge

.github/workflows/ci.yml

@@ -36,9 +36,9 @@ jobs:
           --health-timeout 5s
           --health-retries 3
     steps:
-      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
       - name: setup python
-        uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
+        uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
         with:
           python-version: ${{ matrix.python-version }}
 

.github/workflows/lint-imports.yml

@@ -13,9 +13,9 @@ jobs:
     name: Lint Python Imports
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
       - name: setup python
-        uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
+        uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
         with:
           python-version: '3.12'
 

.github/workflows/pypi-publish.yml

@@ -11,9 +11,9 @@ jobs:
 
     steps:
       - name: Checkout
-        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
       - name: setup python
-        uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
+        uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
         with:
           python-version: 3.12
 

docs/openedx_content/decisions/0005-serving-static-assets.rst

@@ -3,6 +3,11 @@
 5. Serving Course Team Authored Static Assets
 ==============================================
 
+Status
+------
+
+Accepted in principle. Pending implementation.
+
 Context
 --------
 

docs/openedx_learning/decisions/0002-competency-criteria-model.rst

@@ -135,7 +135,7 @@ Decision
    7. ``rule_payload``: JSON payload keyed by ``rule_type`` to avoid freeform strings. It is structured JSON (not arbitrary freeform data): each ``rule_type`` defines the allowed payload shape and required keys, and validation enforces this contract. JSON is used instead of fixed columns like ``op``, ``value``, and ``scale`` so that future rule types (for example, ``MasteryLevel`` thresholds or plugin-defined evaluators such as CEL-based rules) can add their own fields without repeated schema migrations or many nullable columns. Examples:
 
       1. ``Grade``: ``{"op": "gte", "value": 0.75, "scale": "percent"}``. Allowed ``op`` values: ``gte``, ``lte``, ``eq``. ``value`` must be a fraction between 0.0 and 1.0 inclusive, matching the platform's existing fractional grade representation, not a 0-100 scale.
-   8. ``archived``: Boolean, defaults to false. Set instead of deleting a profile that is no longer wanted. Archived profiles are hidden from authoring and new associations but remain queryable, so existing ``CompetencyCriterion`` rows and learner status history stay resolvable.
+   8. ``archived``: Boolean, defaults to false. Set instead of deleting a profile that is no longer wanted. Archived profiles are hidden from authoring and new associations but remain queryable, so existing ``CompetencyCriterion`` rows and learner status rows stay resolvable.
 
    A check constraint requires that at most one of ``organization_id``, ``course_id``, and ``competency_taxonomy_id`` is non-null on any row, matching the scoping rule above.
 
@@ -240,9 +240,9 @@ Decision
    3. ``oel_tagging_objecttag(object_id)``
    4. ``CompetencyCriteria(oel_tagging_objecttag_id)``
    5. ``CompetencyCriteria(competency_criteria_group_id)``
-   6. ``StudentCompetencyCriteriaStatus(user_id, competency_criteria_id)``
-   7. ``StudentCompetencyCriteriaGroupStatus(user_id, competency_criteria_group_id)``
-   8. ``StudentCompetencyStatus(user_id, oel_tagging_tag_id)``
+   6. ``StudentCompetencyCriteriaStatus(user_id, competency_criteria_id)`` (unique -- each learner status table holds exactly one row per learner and target entity, updated in place; see ADR 0003 Decision 5)
+   7. ``StudentCompetencyCriteriaGroupStatus(user_id, competency_criteria_group_id)`` (unique)
+   8. ``StudentCompetencyStatus(user_id, oel_tagging_tag_id)`` (unique)
    9. ``CompetencyRuleProfile(scope_code)`` (unique -- at most one profile per distinct scope value; a plain unique constraint on the three raw nullable scope columns would not enforce this, since SQL never treats two ``NULL`` values as equal and this project's MySQL backend does not support the conditional/partial unique indexes that would otherwise route around that; see the ``scope_code`` column in Decision 3)
    10. ``CompetencyMasteryStatuses(status)`` (unique)
 
@@ -279,23 +279,26 @@ Decision
       2. ``competency_criteria_id``: Foreign key to ``CompetencyCriterion.id``
       3. ``user_id``: Foreign key pointing to user_id (presumably the learner's id, although it appears that it is possible for staff to get grades as well) in ``auth_user`` table
       4. ``status_id``: Foreign key to ``CompetencyMasteryStatuses.id``
-      5. ``created``: The timestamp at which the student's criterion status was set.
+      5. ``created``: The timestamp at which the student's criterion status row was first written.
+      6. ``modified``: The timestamp at which the student's criterion status was last updated.
 
    3. Add a new database table for ``StudentCompetencyCriteriaGroupStatus`` with these columns:
 
       1. ``id``: unique primary key
       2. ``competency_criteria_group_id``: Foreign key to ``CompetencyCriteriaGroup.id``
       3. ``user_id``: Foreign key pointing to user_id (presumably the learner's id, although it appears that it is possible for staff to get grades as well) in ``auth_user`` table
       4. ``status_id``: Foreign key to ``CompetencyMasteryStatuses.id``
-      5. ``created``: The timestamp at which the student's criteria-group status was set.
+      5. ``created``: The timestamp at which the student's criteria-group status row was first written.
+      6. ``modified``: The timestamp at which the student's criteria-group status was last updated.
 
    4. Add a new database table for ``StudentCompetencyStatus`` with these columns:
 
       1. ``id``: unique primary key
       2. ``oel_tagging_tag_id``: Foreign key pointing to Tag id
       3. ``user_id``: Foreign key pointing to user_id (presumably the learner's id, although it appears that it is possible for staff to get grades as well) in ``auth_user`` table
       4. ``status_id``: Foreign key to ``CompetencyMasteryStatuses.id``. This table should have a constraint to only allow status values of “Demonstrated” and “PartiallyAttempted” since it represents overall competency demonstration state, not in-progress states.
-      5. ``created``: The timestamp at which the student's competency status was set.
+      5. ``created``: The timestamp at which the student's competency status row was first written.
+      6. ``modified``: The timestamp at which the student's competency status was last updated.
 
 7. Delete protection boundaries
 
@@ -422,3 +425,11 @@ Rejected Alternatives
 
       1. Silently does not work on this project's tested and production database backend. Django compiles a conditional ``UniqueConstraint`` to a partial index, which MySQL does not support; Django raises only a non-fatal system-check warning (``models.W036``) and skips creating the constraint, leaving the uniqueness rule completely unenforced at the database level.
       2. The gap would surface only as a data-integrity incident under concurrent writes, not as a test or migration failure, since SQLite (used for quick local test runs) does support partial indexes and would mask the problem in that environment.
+
+Changelog
+---------
+
+2026-07-27:
+
+* Made the learner status indexes unique, so there is one row per learner and node. This is what
+  the in-place, monotone status updates in :ref:`openedx-learning-adr-0004` read, lock, and update.

docs/openedx_learning/decisions/0003-competency-criteria-versioning.rst

@@ -13,7 +13,7 @@ Typically, institutions and instructional designers do not change the mastery re
 
 Currently, Open edX always displays the latest edited version of content in the Studio UI and always shows the latest published version of content in the LMS UI, despite having more robust version tracking on the backend (Publishable Entities).
 
-Authoring data (criteria definitions) and runtime learner data (status) have different governance needs. The former is long-lived and typically non-PII, while the latter is user-specific, can be large (learners x criteria/competencies x time), and may require stricter retention and access controls. These differing lifecycles can make deep coupling of authoring and runtime data harder to manage at scale. Performance is also a consideration as computing or resolving versioned criteria for large courses could add overhead in Studio authoring screens or LMS views.
+Authoring data (criteria definitions) and runtime learner data (status) have different governance needs. The former is long-lived and typically non-PII, while the latter is user-specific, can be large (learners x criteria/competencies), and may require stricter retention and access controls. These differing lifecycles can make deep coupling of authoring and runtime data harder to manage at scale. Performance is also a consideration as computing or resolving versioned criteria for large courses could add overhead in Studio authoring screens or LMS views.
 
 Decision
 --------
@@ -44,12 +44,13 @@ For the initial implementation, versioning and traceability of competency achiev
    - A ``CompetencyRuleProfile`` is "in use" if any ``CompetencyCriterion`` assigned to it (``competency_rule_profile_id``) has an associated ``StudentCompetencyCriteriaStatus`` row. Editing an in-use profile's ``rule_type``/``rule_payload`` requires the same warning and confirmation.
    - The same warning applies when creating a more specific profile causes existing criteria to be reassigned to it, and when an authoring action switches a criterion between a profile assignment and per-criterion overrides (ADR 0002 Decision 4).
 
-5. Learner status models/tables are append-only history and do not use ``django-simple-history``:
+5. Do not store history for learner competency status tables, and update rows in place. These tables do not use ``django-simple-history``:
 
-   - For ``StudentCompetencyCriteriaStatus``, ``StudentCompetencyCriteriaGroupStatus``, and ``StudentCompetencyStatus``, each status change is stored as a new row with ``created`` as the write timestamp.
-   - Existing learner status rows are not updated in place.
-   - Current status is determined by the most recent row for a given learner + target entity (ordered by ``created``, with ``id`` as a tie-breaker).
-   - Older rows represent the learner status history and remain available for audit/tracing.
+   - For ``StudentCompetencyCriteriaStatus``, ``StudentCompetencyCriteriaGroupStatus``, and ``StudentCompetencyStatus``, each row is updated in place when a learner's status changes.
+   - There is no history of prior status values beyond the ``modified`` timestamp, and no separate history table.
+   - Current status is the single row for a given learner + target entity.
+
+   Open edX has no concept of gradeable subsection attempts. This means that an attempt is actually defined at the level of an individual problem, so storing one row per attempt can result in tens of billions of rows for a large Open edX instance. That scale creates real operational burden: schema migrations, backups, and truncation and retention policy. Therefore, we did additional market research and found that storing history for learner competency status will not be required by the initial pilot partners for the MVP of the CBE implementation, and it would be safe to add later if needed.
 
 
 Rejected Alternatives
@@ -85,3 +86,43 @@ Rejected Alternatives
     - Cons:
         - Requires custom tooling to reconstruct past versions
         - Does not align with existing publishable versioning patterns
+5. Keep the learner status tables append-only, storing every status change as a new row.
+    - Pros:
+        - Every write is an insert rather than a read-modify-write, so there is no current row to keep consistent.
+        - Preserves a full audit trail of every status change.
+    - Cons:
+        - No MVP requirement calls for this history.
+        - Grows the leaf table by a further multiplier of problem attempts per learner per leaf, reaching tens of billions of rows for a large instance.
+        - A read must resolve the latest row for a learner and node rather than reading one in-place row, which is more expensive and more complex.
+6. Compute leaves transiently, never store them.
+    - Pros:
+        - Eliminates the largest table, since leaf demonstration would be computed on demand from the leaf's rule and the learner's grade.
+    - Cons:
+        - A recomputed leaf reflects the rule as it stands now, not the rule in force when the learner was graded, which contradicts Decision 4 above and can silently lower a status.
+7. Store child evaluations on the parent group row instead of a leaf table.
+    - Pros:
+        - Avoids the largest table entirely.
+    - Cons:
+        - A leaf write becomes a read-modify-write of a column shared with every sibling.
+        - No unique index or foreign key stands behind a status packed into a per-group array, so nothing but application code keeps it consistent with the criteria it describes.
+        - Couples a leaf's frozen mastery to the current shape of the criteria tree, so restructuring the tree can corrupt already-recorded mastery.
+        - Removes the ability to individually track competency status progress by learning object, for example by subsection.
+8. Put the leaf table behind its own database alias and router, a separate physical database, or native partitioning or sharding, from the start.
+    - Pros:
+        - Physically isolates or splits the largest table from the start.
+    - Cons:
+        - A second database alias runs on its own connection, so the leaf write and the ancestor writes could no longer share a single transaction with the grade write, which gives up the atomicity these writes need.
+        - Imposes real operational cost on every deployment with nothing measured to justify it, and remains available later if a specific need is proven.
+9. Serve heavy leaf-table reads from a read replica.
+    - Pros:
+        - Keeps dashboard and reporting reads off the primary.
+    - Cons:
+        - Premature: no measurement shows the primary struggling, and these are point lookups on a composite index, not the wide, expensive reads that drive ``StudentModule`` load in edx-platform.
+        - Adding it later is a per-query choice, not a schema decision.
+10. Give the leaf table a custom unsigned 64-bit primary key.
+     - Cons:
+         - ``BigAutoField``'s range is already far out of reach for this table.
+         - Unsigned integers do not exist in PostgreSQL, and the custom field type carries ongoing maintenance cost for no real benefit at this scale.
+11. Drop the database-level foreign key constraint on the learner column.
+     - Cons:
+         - This repo's convention is a real foreign key to ``settings.AUTH_USER_MODEL``. Reports of contention on the user row exist elsewhere in the ecosystem, but are not understood well enough here to design around.