diff --git a/docs/payout-milestone-migration-runbook.md b/docs/payout-milestone-migration-runbook.md new file mode 100644 index 0000000..4f2c6da --- /dev/null +++ b/docs/payout-milestone-migration-runbook.md @@ -0,0 +1,198 @@ +# Payout Milestone Migration Runbook + +## Overview + +Migration **#5 (`normalize-payout-milestones`)** enhances the existing flat +`milestones` collection with a full document model and adds supporting +collections for evidence and transition history. It also backfills any legacy +milestone data still embedded as a JSON array on `payouts` documents. + +The migration runs in three stages: + +| Stage | What it does | Can be re-run? | +|-------|-------------|----------------| +| **Expand** | Creates `milestone_evidence` and `milestone_history` collections; applies validators and indexes to `milestones` and the new collections. | Yes (idempotent) | +| **Backfill** | Scans existing `milestones` documents and `payouts.milestones` arrays and writes normalised documents to the enhanced `milestones` collection. | Yes (resumable via checkpoint) | +| **Verify** | Counts and sum-checks normalised vs legacy milestone records. Fails loudly on mismatch. | Yes (stateless) | + +## Prerequisites + +- Access to a MongoDB shell (`mongosh`) or Compass for manual verification. +- The `MIGRATION_BACKFILL_BATCH_SIZE` env var (default 100) to control per-checkpoint batch size. +- Sufficient database connection pool for the migration's background scan. + +## Running the Migration + +### 1. Expand (no data changes) + +```bash +node scripts/migrations/migrate-db.mjs --target=5 +``` + +On first run the migration's `up()` function executes **Expand**, then +**Backfill**, then **Verify**. To stop after Expand only, use `--target=4` +(which runs migrations 1–4 but not 5), then later run `--target=5` to +trigger stage 5 only. + +### 2. Backfill (data migration) + +The Backfill stage is **resumable** — if the process crashes partway through, +re-running the migration picks up from the last checkpoint saved in the +`_schema_migrations` document for version 5. + +**Monitoring progress:** + +```javascript +// In mongosh: +use eduvault; +db._schema_migrations.findOne({ version: 5 }, { checkpoint: 1 }); +``` + +The checkpoint field contains: + +```json +{ + "phase": "backfill", + "backfilled": 1423, + "skipped": 2, + "errors": [{ "id": "...", "reason": "...", "collection": "payouts" }], + "processedIds": ["id1", "id2", ...], + "updatedAt": "2026-07-29T..." +} +``` + +If `errors` is non-empty, inspect each entry in the database and decide +whether to: +- Fix the source data and re-run (the migration skips already-processed IDs). +- Manually insert the normalised milestone document. + +### 3. Verify (consistency check) + +Runs automatically after Backfill completes. Fails with a `Verification FAILED` +message if: + +- **Count mismatch**: normalised milestone count < legacy embedded count. +- **Total mismatch**: sum of `amount` fields differs between normalised and + legacy collections (only checked when legacy totals are non-zero). + +The verify stage is **stateless** and can be re-run at any time by adjusting +the checkpoint, or by calling `stageVerify()` directly from a script. + +## Rollback + +### Mid-Expand (safe) + +If the migration crashes during the Expand stage, no data has been changed. +Simply re-run. + +### Mid-Backfill (safe, resumable) + +If the migration crashes during Backfill: + +1. Confirm the checkpoint was saved by checking `_schema_migrations`. +2. Fix any data issues reported in `checkpoint.errors`. +3. **Re-run the migration** — it resumes from the checkpoint. + +To roll back changes made by a partial backfill: + +```bash +# 1. Drop new collections +mongosh eduvault --eval 'db.milestone_evidence.drop(); db.milestone_history.drop();' +# 2. Remove enhanced indexes from milestones +mongosh eduvault --eval ' + db.milestones.dropIndex("milestones_payout_id"); + db.milestones.dropIndex("milestones_payout_order_unique"); + db.milestones.dropIndex("milestones_status_payout"); +' +# 3. Reset migration status for version 5: +db._schema_migrations.deleteOne({ version: 5 }); +``` + +### Full Rollback (after successful migration) + +The `down()` function of migration 005 removes: +- The `milestone_evidence` and `milestone_history` collections. +- The enhanced indexes on `milestones`. + +To roll back: + +```bash +# Manually run the down logic (the migration runner does not support +# automatic down migrations from the CLI). Use mongosh: +mongosh eduvault --eval ' + db.milestone_evidence.drop(); + db.milestone_history.drop(); + db.milestones.dropIndex("milestones_payout_id"); + db.milestones.dropIndex("milestones_payout_order_unique"); + db.milestones.dropIndex("milestones_status_payout"); +' +# Remove the migration record so it can be re-applied: +db._schema_migrations.deleteOne({ version: 5 }); +``` + +### Data Reversal (restore legacy embedded milestones) + +If you need to restore the legacy `milestones` array on payout documents from +the normalised collection: + +```javascript +const milestones = db.milestones.find({ payoutId: { $exists: true } }).toArray(); +const byPayout = {}; +for (const m of milestones) { + if (!byPayout[m.payoutId]) byPayout[m.payoutId] = []; + byPayout[m.payoutId].push({ + milestoneId: m.milestoneId, + order: m.order, + title: m.title, + description: m.description, + amount: m.amount, + status: m.status, + }); +} +for (const [payoutId, ms] of Object.entries(byPayout)) { + db.payouts.updateOne({ payoutId }, { $set: { milestones: ms } }); +} +``` + +## Dual-Read / Dual-Write Strategy + +**Design choice: dual-read, not dual-write.** + +The `milestoneService.js` layer: +- **Reads** from the normalised `milestones` collection first; falls back to + the legacy `payouts.milestones` embedded array if no normalised records exist. +- **Writes** to the normalised `milestones` collection and **also syncs** the + legacy `payouts.milestones` field so readers still using the old path see + current data. + +This makes the rollout: +- **Reversible** at any point (downgraded code sees the legacy field). +- **Safe for partial deployment** (some instances can run new code, others old). + +## Contract Phase (remove legacy field) + +After the migration has been verified in production for at least one release +cycle, a follow-up migration (e.g. version 6) should: + +1. Confirm all milestones are in the normalised collection (count check). +2. `$unset` the `milestones` field from every `payouts` document. +3. Remove the legacy fallback paths from `milestoneService.js`. +4. Update the `payouts` validator in `schemaContracts.js` to no longer allow + the `milestones` field. + +This follow-up migration is intentionally **not** included in version 5 so that +deployment can be paused or reverted at any point without data loss. + +## Monitoring & Alerts + +- Check `_schema_migrations` for `{ version: 5, status: "failed" }`. +- Alert on `checkpoint.errors` length > 0. +- After Verify, confirm `checkpoint` is deleted (means all stages completed). + +## Troubleshooting + +| Symptom | Likely Cause | Fix | +|---------|-------------|-----| +| `E11000 duplicate key` during backfill | Two workers running concurrently | Re-run; the migration skips already-processed IDs | +| `Verification FAILED` | Count or total mismatch | Inspect checkpoint.errors; fix source data; re-run | +| Migration lock held | Another process has the lock | Wait 60s for expiry, or kill the other process | diff --git a/src/app/api/escrows/[escrowId]/milestones/route.js b/src/app/api/escrows/[escrowId]/milestones/route.js new file mode 100644 index 0000000..bc26a2a --- /dev/null +++ b/src/app/api/escrows/[escrowId]/milestones/route.js @@ -0,0 +1,71 @@ +export const dynamic = "force-dynamic"; + +import { NextResponse } from "next/server"; +import { withApiHardening } from "@/lib/api/hardening"; +import { withAuthorization } from "@/lib/auth/authorize"; +import { getDb } from "@/lib/mongodb"; +import { + getMilestonesByEscrow, + createMilestone, +} from "@/lib/backend/milestoneService"; + +export const GET = withApiHardening( + withAuthorization( + async (authorizedRequest, { params }) => { + try { + const { escrowId } = await params; + if (!escrowId) { + return NextResponse.json({ error: "Escrow ID is required" }, { status: 400 }); + } + + const db = await getDb(); + const milestones = await getMilestonesByEscrow(db, escrowId); + + return NextResponse.json(milestones); + } catch (err) { + console.error("[api/escrows/milestones] GET error:", err); + return NextResponse.json({ error: "Server error" }, { status: 500 }); + } + }, + { public: true }, + ), + { route: "escrow_milestones", rateLimit: { limit: 120, windowMs: 60_000 } }, +); + +export const POST = withApiHardening( + withAuthorization( + async (authorizedRequest, { params }) => { + try { + const { escrowId } = await params; + if (!escrowId) { + return NextResponse.json({ error: "Escrow ID is required" }, { status: 400 }); + } + + const body = await authorizedRequest.json(); + const db = await getDb(); + + const milestone = await createMilestone(db, { + payoutId: body.payoutId || null, + escrowId, + order: body.order, + title: body.title, + description: body.description, + amount: body.amount, + currency: body.currency, + dueDate: body.dueDate ? new Date(body.dueDate) : null, + createdBy: authorizedRequest.userId || body.createdBy || null, + }); + + return NextResponse.json(milestone, { status: 201 }); + } catch (err) { + if (err.message?.includes("exceeds payout amount")) { + return NextResponse.json({ error: err.message }, { status: 422 }); + } + console.error("[api/escrows/milestones] POST error:", err); + return NextResponse.json({ error: "Server error" }, { status: 500 }); + } + }, + {}, + ), + { route: "escrow_milestones_create", rateLimit: { limit: 30, windowMs: 60_000 } }, +); diff --git a/src/app/api/milestones/[milestoneId]/approve/route.js b/src/app/api/milestones/[milestoneId]/approve/route.js new file mode 100644 index 0000000..a7131d3 --- /dev/null +++ b/src/app/api/milestones/[milestoneId]/approve/route.js @@ -0,0 +1,39 @@ +export const dynamic = "force-dynamic"; + +import { NextResponse } from "next/server"; +import { withApiHardening } from "@/lib/api/hardening"; +import { withAuthorization } from "@/lib/auth/authorize"; +import { getDb } from "@/lib/mongodb"; +import { transitionMilestoneStatus } from "@/lib/backend/milestoneService"; + +export const POST = withApiHardening( + withAuthorization( + async (authorizedRequest, { params }) => { + try { + const { milestoneId } = await params; + if (!milestoneId) { + return NextResponse.json({ error: "Milestone ID is required" }, { status: 400 }); + } + + const body = await authorizedRequest.json().catch(() => ({})); + const db = await getDb(); + + const result = await transitionMilestoneStatus(db, milestoneId, "approved", { + changedBy: authorizedRequest.userId || body.changedBy || "unknown", + reason: body.reason || null, + chainTxHash: body.chainTxHash || null, + }); + + return NextResponse.json(result); + } catch (err) { + if (err.message?.includes("not found")) { + return NextResponse.json({ error: err.message }, { status: 404 }); + } + console.error("[api/milestones/approve] POST error:", err); + return NextResponse.json({ error: "Server error" }, { status: 500 }); + } + }, + {}, + ), + { route: "milestone_approve", rateLimit: { limit: 30, windowMs: 60_000 } }, +); diff --git a/src/app/api/milestones/[milestoneId]/evidence/route.js b/src/app/api/milestones/[milestoneId]/evidence/route.js new file mode 100644 index 0000000..4cfb4da --- /dev/null +++ b/src/app/api/milestones/[milestoneId]/evidence/route.js @@ -0,0 +1,64 @@ +export const dynamic = "force-dynamic"; + +import { NextResponse } from "next/server"; +import { withApiHardening } from "@/lib/api/hardening"; +import { withAuthorization } from "@/lib/auth/authorize"; +import { getDb } from "@/lib/mongodb"; +import { + addMilestoneEvidence, + getMilestoneEvidence, +} from "@/lib/backend/milestoneService"; + +export const GET = withApiHardening( + withAuthorization( + async (authorizedRequest, { params }) => { + try { + const { milestoneId } = await params; + if (!milestoneId) { + return NextResponse.json({ error: "Milestone ID is required" }, { status: 400 }); + } + + const db = await getDb(); + const evidence = await getMilestoneEvidence(db, milestoneId); + + return NextResponse.json(evidence); + } catch (err) { + console.error("[api/milestones/evidence] GET error:", err); + return NextResponse.json({ error: "Server error" }, { status: 500 }); + } + }, + { public: true }, + ), + { route: "milestone_evidence", rateLimit: { limit: 120, windowMs: 60_000 } }, +); + +export const POST = withApiHardening( + withAuthorization( + async (authorizedRequest, { params }) => { + try { + const { milestoneId } = await params; + if (!milestoneId) { + return NextResponse.json({ error: "Milestone ID is required" }, { status: 400 }); + } + + const body = await authorizedRequest.json(); + const db = await getDb(); + + const evidence = await addMilestoneEvidence(db, milestoneId, { + uploadedBy: authorizedRequest.userId || body.uploadedBy || "unknown", + fileId: body.fileId || null, + fileUrl: body.fileUrl || null, + fileType: body.fileType || null, + notes: body.notes || null, + }); + + return NextResponse.json(evidence, { status: 201 }); + } catch (err) { + console.error("[api/milestones/evidence] POST error:", err); + return NextResponse.json({ error: "Server error" }, { status: 500 }); + } + }, + {}, + ), + { route: "milestone_evidence_create", rateLimit: { limit: 30, windowMs: 60_000 } }, +); diff --git a/src/app/api/milestones/[milestoneId]/reject/route.js b/src/app/api/milestones/[milestoneId]/reject/route.js new file mode 100644 index 0000000..634411d --- /dev/null +++ b/src/app/api/milestones/[milestoneId]/reject/route.js @@ -0,0 +1,38 @@ +export const dynamic = "force-dynamic"; + +import { NextResponse } from "next/server"; +import { withApiHardening } from "@/lib/api/hardening"; +import { withAuthorization } from "@/lib/auth/authorize"; +import { getDb } from "@/lib/mongodb"; +import { transitionMilestoneStatus } from "@/lib/backend/milestoneService"; + +export const POST = withApiHardening( + withAuthorization( + async (authorizedRequest, { params }) => { + try { + const { milestoneId } = await params; + if (!milestoneId) { + return NextResponse.json({ error: "Milestone ID is required" }, { status: 400 }); + } + + const body = await authorizedRequest.json().catch(() => ({})); + const db = await getDb(); + + const result = await transitionMilestoneStatus(db, milestoneId, "rejected", { + changedBy: authorizedRequest.userId || body.changedBy || "unknown", + reason: body.reason || null, + }); + + return NextResponse.json(result); + } catch (err) { + if (err.message?.includes("not found")) { + return NextResponse.json({ error: err.message }, { status: 404 }); + } + console.error("[api/milestones/reject] POST error:", err); + return NextResponse.json({ error: "Server error" }, { status: 500 }); + } + }, + {}, + ), + { route: "milestone_reject", rateLimit: { limit: 30, windowMs: 60_000 } }, +); diff --git a/src/app/api/milestones/[milestoneId]/route.js b/src/app/api/milestones/[milestoneId]/route.js new file mode 100644 index 0000000..2bdf970 --- /dev/null +++ b/src/app/api/milestones/[milestoneId]/route.js @@ -0,0 +1,95 @@ +export const dynamic = "force-dynamic"; + +import { NextResponse } from "next/server"; +import { withApiHardening } from "@/lib/api/hardening"; +import { withAuthorization } from "@/lib/auth/authorize"; +import { getDb } from "@/lib/mongodb"; +import { + getMilestoneById, + updateMilestone, + deleteMilestone, +} from "@/lib/backend/milestoneService"; + +export const GET = withApiHardening( + withAuthorization( + async (authorizedRequest, { params }) => { + try { + const { milestoneId } = await params; + if (!milestoneId) { + return NextResponse.json({ error: "Milestone ID is required" }, { status: 400 }); + } + + const db = await getDb(); + const milestone = await getMilestoneById(db, milestoneId); + if (!milestone) { + return NextResponse.json({ error: "Milestone not found" }, { status: 404 }); + } + return NextResponse.json(milestone); + } catch (err) { + console.error("[api/milestones] GET error:", err); + return NextResponse.json({ error: "Server error" }, { status: 500 }); + } + }, + { public: true }, + ), + { route: "milestone_detail", rateLimit: { limit: 120, windowMs: 60_000 } }, +); + +export const PUT = withApiHardening( + withAuthorization( + async (authorizedRequest, { params }) => { + try { + const { milestoneId } = await params; + if (!milestoneId) { + return NextResponse.json({ error: "Milestone ID is required" }, { status: 400 }); + } + + const body = await authorizedRequest.json(); + const db = await getDb(); + + const expectedVersion = body.version; + const updateFields = { ...body }; + delete updateFields.version; + + const updated = await updateMilestone(db, milestoneId, updateFields, expectedVersion); + return NextResponse.json(updated); + } catch (err) { + if (err.message?.includes("version conflict")) { + return NextResponse.json({ error: err.message }, { status: 409 }); + } + if (err.message?.includes("not found")) { + return NextResponse.json({ error: err.message }, { status: 404 }); + } + console.error("[api/milestones] PUT error:", err); + return NextResponse.json({ error: "Server error" }, { status: 500 }); + } + }, + {}, + ), + { route: "milestone_update", rateLimit: { limit: 60, windowMs: 60_000 } }, +); + +export const DELETE = withApiHardening( + withAuthorization( + async (authorizedRequest, { params }) => { + try { + const { milestoneId } = await params; + if (!milestoneId) { + return NextResponse.json({ error: "Milestone ID is required" }, { status: 400 }); + } + + const db = await getDb(); + const deleted = await deleteMilestone(db, milestoneId); + if (!deleted) { + return NextResponse.json({ error: "Milestone not found" }, { status: 404 }); + } + return NextResponse.json({ deleted: true }); + } catch (err) { + console.error("[api/milestones] DELETE error:", err); + return NextResponse.json({ error: "Server error" }, { status: 500 }); + } + }, + {}, + ), + { route: "milestone_delete", rateLimit: { limit: 30, windowMs: 60_000 } }, +); diff --git a/src/app/api/milestones/[milestoneId]/submit/route.js b/src/app/api/milestones/[milestoneId]/submit/route.js new file mode 100644 index 0000000..e6491f8 --- /dev/null +++ b/src/app/api/milestones/[milestoneId]/submit/route.js @@ -0,0 +1,38 @@ +export const dynamic = "force-dynamic"; + +import { NextResponse } from "next/server"; +import { withApiHardening } from "@/lib/api/hardening"; +import { withAuthorization } from "@/lib/auth/authorize"; +import { getDb } from "@/lib/mongodb"; +import { transitionMilestoneStatus } from "@/lib/backend/milestoneService"; + +export const POST = withApiHardening( + withAuthorization( + async (authorizedRequest, { params }) => { + try { + const { milestoneId } = await params; + if (!milestoneId) { + return NextResponse.json({ error: "Milestone ID is required" }, { status: 400 }); + } + + const body = await authorizedRequest.json().catch(() => ({})); + const db = await getDb(); + + const result = await transitionMilestoneStatus(db, milestoneId, "submitted", { + changedBy: authorizedRequest.userId || body.changedBy || "unknown", + reason: body.reason || null, + }); + + return NextResponse.json(result); + } catch (err) { + if (err.message?.includes("not found")) { + return NextResponse.json({ error: err.message }, { status: 404 }); + } + console.error("[api/milestones/submit] POST error:", err); + return NextResponse.json({ error: "Server error" }, { status: 500 }); + } + }, + {}, + ), + { route: "milestone_submit", rateLimit: { limit: 30, windowMs: 60_000 } }, +); diff --git a/src/components/milestones/ManageMilestonesDialog.jsx b/src/components/milestones/ManageMilestonesDialog.jsx new file mode 100644 index 0000000..ca9c96e --- /dev/null +++ b/src/components/milestones/ManageMilestonesDialog.jsx @@ -0,0 +1,164 @@ +"use client"; + +import { useState } from "react"; +import Modal from "@/components/Modal"; +import { useEscrowMilestones } from "@/hooks/useEscrow"; +import { useMilestoneActions } from "@/hooks/useMilestoneActions"; + +const STATUS_BADGE = { + pending: "bg-yellow-100 text-yellow-800", + submitted: "bg-blue-100 text-blue-800", + approved: "bg-green-100 text-green-800", + rejected: "bg-red-100 text-red-800", + completed: "bg-gray-100 text-gray-800", +}; + +function StatusBadge({ status }) { + const classes = STATUS_BADGE[status] || "bg-gray-100 text-gray-800"; + return ( + + {status} + + ); +} + +export default function ManageMilestonesDialog({ escrowId, payoutId, isOpen, onClose }) { + const { data: milestones = [], isLoading } = useEscrowMilestones(escrowId, { enabled: isOpen && !!escrowId }); + const actions = useMilestoneActions(escrowId); + + const [newMilestone, setNewMilestone] = useState({ title: "", amount: "", dueDate: "" }); + const [actionLoading, setActionLoading] = useState(null); + + async function handleCreate() { + if (!newMilestone.title.trim()) return; + setActionLoading("create"); + try { + await actions.createMilestone.mutateAsync({ + payoutId, + title: newMilestone.title, + amount: newMilestone.amount, + dueDate: newMilestone.dueDate || undefined, + }); + setNewMilestone({ title: "", amount: "", dueDate: "" }); + } finally { + setActionLoading(null); + } + } + + async function handleStatusChange(milestoneId, action, extra = {}) { + setActionLoading(`${action}-${milestoneId}`); + try { + const fn = actions[`${action}Milestone`]; + if (fn) await fn.mutateAsync({ milestoneId, ...extra }); + } finally { + setActionLoading(null); + } + } + + return ( + + {isLoading ? ( +

Loading milestones...

+ ) : ( +
+ {milestones.length === 0 && ( +

No milestones yet. Create one below.

+ )} + + {milestones.map((milestone) => ( +
+
+ + {milestone.order != null && `#${milestone.order} `} + {milestone.title || milestone.description || "Untitled"} + + +
+ + {milestone.amount && ( +

+ Amount: {milestone.amount} {milestone.currency || ""} +

+ )} + + {milestone.dueDate && ( +

+ Due: {new Date(milestone.dueDate).toLocaleDateString()} +

+ )} + + {milestone.feedback && ( +

Feedback: {milestone.feedback}

+ )} + +
+ {milestone.status === "pending" && ( + + )} + {milestone.status === "submitted" && ( + <> + + + + )} +
+
+ ))} + +
+

Add Milestone

+ setNewMilestone((prev) => ({ ...prev, title: e.target.value }))} + className="w-full border rounded px-2 py-1 text-sm" + /> + setNewMilestone((prev) => ({ ...prev, amount: e.target.value }))} + className="w-full border rounded px-2 py-1 text-sm" + /> + setNewMilestone((prev) => ({ ...prev, dueDate: e.target.value }))} + className="w-full border rounded px-2 py-1 text-sm" + /> + +
+
+ )} + + ); +} diff --git a/src/hooks/useMilestoneActions.js b/src/hooks/useMilestoneActions.js new file mode 100644 index 0000000..2a5c631 --- /dev/null +++ b/src/hooks/useMilestoneActions.js @@ -0,0 +1,73 @@ +import { useMutation, useQueryClient } from "@tanstack/react-query"; + +async function apiRequest(url, method, body) { + const options = { + method, + headers: { "Content-Type": "application/json" }, + }; + if (body !== undefined) options.body = JSON.stringify(body); + + const res = await fetch(url, options); + const data = await res.json(); + if (!res.ok) throw new Error(data.error || `Request failed with status ${res.status}`); + return data; +} + +export function useMilestoneActions(escrowId, { onSuccess, onError } = {}) { + const queryClient = useQueryClient(); + + const invalidate = () => { + queryClient.invalidateQueries({ queryKey: ["escrow", escrowId, "milestones"] }); + queryClient.invalidateQueries({ queryKey: ["escrow", escrowId] }); + }; + + const createMilestone = useMutation({ + mutationFn: (data) => apiRequest(`/api/escrows/${escrowId}/milestones`, "POST", data), + onSuccess: () => { invalidate(); onSuccess?.("Milestone created"); }, + onError, + }); + + const updateMilestone = useMutation({ + mutationFn: ({ milestoneId, ...data }) => + apiRequest(`/api/milestones/${milestoneId}`, "PUT", data), + onSuccess: () => { invalidate(); onSuccess?.("Milestone updated"); }, + onError, + }); + + const submitMilestone = useMutation({ + mutationFn: ({ milestoneId, reason }) => + apiRequest(`/api/milestones/${milestoneId}/submit`, "POST", { reason }), + onSuccess: () => { invalidate(); onSuccess?.("Milestone submitted"); }, + onError, + }); + + const approveMilestone = useMutation({ + mutationFn: ({ milestoneId, reason, chainTxHash }) => + apiRequest(`/api/milestones/${milestoneId}/approve`, "POST", { reason, chainTxHash }), + onSuccess: () => { invalidate(); onSuccess?.("Milestone approved"); }, + onError, + }); + + const rejectMilestone = useMutation({ + mutationFn: ({ milestoneId, reason }) => + apiRequest(`/api/milestones/${milestoneId}/reject`, "POST", { reason }), + onSuccess: () => { invalidate(); onSuccess?.("Milestone rejected"); }, + onError, + }); + + const addEvidence = useMutation({ + mutationFn: ({ milestoneId, ...data }) => + apiRequest(`/api/milestones/${milestoneId}/evidence`, "POST", data), + onSuccess: () => { invalidate(); onSuccess?.("Evidence added"); }, + onError, + }); + + return { + createMilestone, + updateMilestone, + submitMilestone, + approveMilestone, + rejectMilestone, + addEvidence, + }; +} diff --git a/src/lib/backend/migrations/005-normalize-payout-milestones.js b/src/lib/backend/migrations/005-normalize-payout-milestones.js new file mode 100644 index 0000000..f6302c5 --- /dev/null +++ b/src/lib/backend/migrations/005-normalize-payout-milestones.js @@ -0,0 +1,401 @@ +import process from "node:process"; + +import { + COLLECTION_VALIDATORS, + COLLECTIONS, + REQUIRED_INDEXES, +} from "../schemaContracts.js"; + +const MILESTONE_COLLECTION = COLLECTIONS.milestones; +const EVIDENCE_COLLECTION = COLLECTIONS.milestoneEvidence || "milestone_evidence"; +const HISTORY_COLLECTION = COLLECTIONS.milestoneHistory || "milestone_history"; +const PAYOUT_COLLECTION = COLLECTIONS.payouts; + +const BACKFILL_BATCH_SIZE = Number.parseInt( + process.env.MIGRATION_BACKFILL_BATCH_SIZE || "100", + 10, +); + +// ── helpers ────────────────────────────────────────────────────────────────── + +async function collectionExists(db, name) { + const collections = await db.listCollections({ name }, { nameOnly: true }).toArray(); + return collections.length > 0; +} + +async function ensureCollection(db, name) { + if (!(await collectionExists(db, name))) { + await db.createCollection(name); + } +} + +function normaliseMilestone(raw, payoutId, escrowId) { + if (!raw || typeof raw !== "object") return null; + + const milestoneId = raw.milestoneId || raw.id || null; + if (!milestoneId) return null; + + const amount = raw.amount != null ? String(raw.amount) : null; + + let dueDate = null; + if (raw.dueDate) { + dueDate = raw.dueDate instanceof Date ? raw.dueDate : new Date(raw.dueDate); + if (Number.isNaN(dueDate.getTime())) dueDate = null; + } + + let createdAt = raw.createdAt instanceof Date ? raw.createdAt : raw.createdAt ? new Date(raw.createdAt) : null; + if (createdAt && Number.isNaN(createdAt.getTime())) createdAt = null; + + return { + milestoneId, + payoutId: payoutId || null, + escrowId: escrowId || raw.escrowId || "", + order: raw.order != null ? raw.order : 0, + title: raw.title || raw.description || "", + description: raw.description || null, + amount, + currency: raw.currency || null, + dueDate, + status: raw.status || "pending", + evidenceIds: Array.isArray(raw.evidenceIds) ? raw.evidenceIds : [], + feedback: raw.feedback || null, + chainTxHash: raw.chainTxHash || null, + createdBy: raw.createdBy || null, + version: 1, + createdAt: createdAt || new Date(0), + updatedAt: new Date(), + }; +} + +// ── stages ─────────────────────────────────────────────────────────────────── + +/** + * Stage 1 — Expand: create new collections and indexes alongside existing + * data without touching anything. + */ +async function stageExpand({ db, logger }) { + const newCollections = [EVIDENCE_COLLECTION, HISTORY_COLLECTION]; + + for (const name of newCollections) { + await ensureCollection(db, name); + logger.info?.("[migration:005] Collection ensured", { collectionName: name }); + } + + // Ensure the milestones collection itself exists (it should from migration 001, + // but be safe). + if (!(await collectionExists(db, MILESTONE_COLLECTION))) { + await db.createCollection(MILESTONE_COLLECTION); + logger.info?.("[migration:005] Milestones collection created"); + } + + // Apply validators (moderate level to allow legacy docs through). + const validatorCollections = [MILESTONE_COLLECTION, EVIDENCE_COLLECTION, HISTORY_COLLECTION]; + for (const name of validatorCollections) { + const validator = COLLECTION_VALIDATORS[name]; + if (validator) { + try { + await db.command({ + collMod: name, + validator, + validationLevel: "moderate", + validationAction: "error", + }); + logger.info?.("[migration:005] Validator applied", { collectionName: name }); + } catch (err) { + // Collection might not exist in COLLECTION_VALIDATORS yet. + logger.warn?.("[migration:005] Validator skipped", { collectionName: name, reason: err.message }); + } + } + } + + // Create indexes (safe — createIndexes is idempotent). + const indexTargets = [MILESTONE_COLLECTION, EVIDENCE_COLLECTION, HISTORY_COLLECTION]; + for (const name of indexTargets) { + const defs = REQUIRED_INDEXES[name]; + if (defs?.length) { + const models = defs.map((d) => ({ + key: d.keys, + name: d.name, + ...d.options, + })); + await db.collection(name).createIndexes(models); + logger.info?.("[migration:005] Indexes created", { collectionName: name, count: models.length }); + } + } + + logger.info?.("[migration:005] Expand stage complete"); +} + +/** + * Stage 2 — Backfill: migrate existing milestone data from two sources: + * 1. Existing bare-bones milestone documents in the milestones collection. + * 2. Legacy embedded `milestones` arrays on payout documents. + * + * Tracks processed milestoneIds in a checkpoint so the migration is + * idempotent and resumable after interruption. + */ +async function stageBackfill({ db, logger, checkpoint, saveCheckpoint }) { + let processed = checkpoint?.backfilled ?? 0; + let skipped = checkpoint?.skipped ?? 0; + let errors = checkpoint?.errors ?? []; + + // ── Source A: existing milestones collection ─────────────────────────── + const milestonesColl = db.collection(MILESTONE_COLLECTION); + const alreadyProcessed = new Set(checkpoint?.processedIds ?? []); + + const existingDocs = await milestonesColl.find({}).toArray(); + for (const doc of existingDocs) { + if (alreadyProcessed.has(doc.milestoneId || doc._id)) continue; + + try { + // If the doc already has the new fields (evidenceIds, version, etc.), + // skip it (already migrated). + if (doc.version && doc.evidenceIds !== undefined) { + alreadyProcessed.add(doc.milestoneId || doc._id); + processed += 1; + continue; + } + + const normalised = normaliseMilestone( + doc, + doc.payoutId || null, + doc.escrowId, + ); + + if (!normalised) { + skipped += 1; + errors.push({ id: doc._id, reason: "malformed source document", collection: MILESTONE_COLLECTION }); + continue; + } + + // Use the existing _id so the document is replaced in-place. + await milestonesColl.replaceOne( + { _id: doc._id }, + { ...normalised, _id: doc._id }, + { upsert: false }, + ); + + alreadyProcessed.add(normalised.milestoneId); + processed += 1; + } catch (err) { + skipped += 1; + errors.push({ id: doc._id, reason: err.message, collection: MILESTONE_COLLECTION }); + } + } + + // ── Source B: legacy embedded milestones on payouts ──────────────────── + const payoutsColl = db.collection(PAYOUT_COLLECTION); + const cursor = payoutsColl.find({ + milestones: { $exists: true, $ne: null, $type: "array" }, + }); + + for await (const payout of cursor) { + if (!Array.isArray(payout.milestones) || payout.milestones.length === 0) continue; + + for (const raw of payout.milestones) { + const legacyId = raw.milestoneId || raw.id; + if (!legacyId) { + skipped += 1; + errors.push({ id: payout._id, reason: "legacy milestone missing identifier", collection: PAYOUT_COLLECTION }); + continue; + } + + if (alreadyProcessed.has(legacyId)) continue; + + try { + // Check if already exists in normalised collection. + const existing = await milestonesColl.findOne({ milestoneId: legacyId }); + if (existing) { + alreadyProcessed.add(legacyId); + processed += 1; + continue; + } + + const normalised = normaliseMilestone(raw, payout.payoutId, payout.escrowId); + if (!normalised) { + skipped += 1; + errors.push({ id: legacyId, reason: "malformed legacy milestone", collection: PAYOUT_COLLECTION }); + continue; + } + + await milestonesColl.insertOne(normalised); + alreadyProcessed.add(legacyId); + processed += 1; + } catch (err) { + if (err?.code === 11000) { + // Duplicate key — already migrated by another worker. + alreadyProcessed.add(legacyId); + processed += 1; + } else { + skipped += 1; + errors.push({ id: legacyId, reason: err.message, collection: PAYOUT_COLLECTION }); + } + } + } + } + + // ── Save checkpoint ──────────────────────────────────────────────────── + const nextCheckpoint = { + phase: "backfill", + backfilled: processed, + skipped, + errors: errors.slice(0, 1000), // cap to avoid bloating the checkpoint doc + processedIds: [...alreadyProcessed].slice(0, 10000), // cap + updatedAt: new Date(), + }; + + await saveCheckpoint(nextCheckpoint); + + logger.info?.("[migration:005] Backfill stage complete", { + processed, + skipped, + errorCount: errors.length, + }); + + return { processed, skipped, errors }; +} + +/** + * Stage 3 — Verify: count and sum-check source vs. target. + */ +async function stageVerify({ db, logger }) { + const milestonesColl = db.collection(MILESTONE_COLLECTION); + const payoutsColl = db.collection(PAYOUT_COLLECTION); + + // Count milestones in the normalised collection. + const normalisedCount = await milestonesColl.countDocuments({}); + + // Count milestones that existed or exist as embedded arrays on payouts. + let legacyCount = 0; + const payoutsWithMilestones = payoutsColl.find({ + milestones: { $exists: true, $ne: null, $type: "array" }, + }); + for await (const payout of payoutsWithMilestones) { + legacyCount += (payout.milestones || []).length; + } + + // Sum-check: total milestone amounts vs. payout amounts. + // Aggregation on the normalised collection. + let normalisedTotal = "0"; + try { + const aggResult = await milestonesColl.aggregate([ + { $match: { amount: { $exists: true, $ne: null } } }, + { $group: { _id: null, total: { $sum: { $toDecimal: "$amount" } } } }, + ]).toArray(); + normalisedTotal = aggResult[0]?.total?.toString() || "0"; + } catch { + // If amount fields are not all numeric, the $toDecimal will fail. + // Fall back to string summation. + logger.warn?.("[migration:005] Decimal aggregation failed, falling back to string parse"); + const allDocs = await milestonesColl.find({ amount: { $exists: true, $ne: null } }, { projection: { amount: 1 } }).toArray(); + normalisedTotal = String(allDocs.reduce((sum, d) => sum + Number(d.amount || 0), 0)); + } + + // Sum-check on legacy. + let legacyTotal = "0"; + const legacyPayouts = payoutsColl.find({ + milestones: { $exists: true, $ne: null, $type: "array" }, + }); + for await (const payout of legacyPayouts) { + for (const m of (payout.milestones || [])) { + legacyTotal = String(Number(legacyTotal) + Number(m.amount || 0)); + } + } + + const checks = { + normalisedCount, + legacyCount, + normalisedTotal, + legacyTotal, + }; + + logger.info?.("[migration:005] Verify stage results", checks); + + // Fail loudly on mismatches. + if (normalisedCount < legacyCount) { + throw new Error( + `Verification FAILED: normalised count (${normalisedCount}) < legacy count (${legacyCount}). ` + + "Some milestones were not migrated.", + ); + } + + // Only compare totals if there are legacy amounts. + if (legacyTotal !== "0" && normalisedTotal !== legacyTotal) { + throw new Error( + `Verification FAILED: total amount mismatch — normalised ${normalisedTotal} vs legacy ${legacyTotal}.`, + ); + } + + logger.info?.("[migration:005] Verify stage PASSED"); + return checks; +} + +// ── exports (for testing) ───────────────────────────────────────────────────── + +export { normaliseMilestone, stageExpand, stageBackfill, stageVerify }; + +// ── migration export ───────────────────────────────────────────────────────── + +const migration = { + version: 5, + name: "normalize-payout-milestones", + description: + "Expands the milestones collection with full document model, adds milestone_evidence and " + + "milestone_history collections, backfills legacy embedded milestones, and verifies consistency.", + + async up({ db, logger = console, getCheckpoint, saveCheckpoint, clearCheckpoint }) { + const existingCheckpoint = (await getCheckpoint()) || {}; + const phase = existingCheckpoint.phase || null; + + // Expand — run once, no checkpoint needed (idempotent). + if (!phase || phase === "expand") { + logger.info?.("[migration:005] Starting expand stage"); + await stageExpand({ db, logger }); + await saveCheckpoint({ phase: "backfill", backfilled: 0, skipped: 0, errors: [], processedIds: [], updatedAt: new Date() }); + } + + // Backfill — checkpoint-driven, resumable. + const backfillCheckpoint = await getCheckpoint(); + if (backfillCheckpoint?.phase === "backfill") { + logger.info?.("[migration:005] Starting backfill stage"); + await stageBackfill({ db, logger, checkpoint: backfillCheckpoint, saveCheckpoint }); + } + + // Verify — runs after backfill completes. + logger.info?.("[migration:005] Starting verify stage"); + await stageVerify({ db, logger }); + + await clearCheckpoint(); + logger.info?.("[migration:005] Migration complete"); + }, + + async down({ db, logger = console }) { + // Remove new collections. + for (const name of [EVIDENCE_COLLECTION, HISTORY_COLLECTION]) { + if (await collectionExists(db, name)) { + await db.collection(name).drop(); + logger.info?.("[migration:005] Dropped collection", { collectionName: name }); + } + } + + // Remove enhanced indexes from milestones (the base ones from migration 001 stay). + const extraIndexes = [ + "milestones_payout_id", + "milestones_payout_order_unique", + "milestones_status_payout", + ]; + const milestonesColl = db.collection(MILESTONE_COLLECTION); + for (const indexName of extraIndexes) { + try { + await milestonesColl.dropIndex(indexName); + logger.info?.("[migration:005] Dropped index", { indexName }); + } catch { + // May not exist. + } + } + + logger.info?.("[migration:005] Rollback complete"); + }, +}; + +export default migration; diff --git a/src/lib/backend/migrations/registry.js b/src/lib/backend/migrations/registry.js index 860267a..3130b83 100644 --- a/src/lib/backend/migrations/registry.js +++ b/src/lib/backend/migrations/registry.js @@ -2,12 +2,14 @@ import migration001 from "./001-initialize-schema.js"; import migration002 from "./002-resolve-legacy-duplicates.js"; import migration003 from "./003-enforce-unique-indexes.js"; import migration004 from "./004-material-lifecycle.js"; +import migration005 from "./005-normalize-payout-milestones.js"; export const MIGRATIONS = Object.freeze([ migration001, migration002, migration003, migration004, + migration005, ]); export function validateMigrationRegistry( diff --git a/src/lib/backend/milestoneService.js b/src/lib/backend/milestoneService.js new file mode 100644 index 0000000..866eb0e --- /dev/null +++ b/src/lib/backend/milestoneService.js @@ -0,0 +1,393 @@ +import { randomUUID } from "node:crypto"; + +import { COLLECTIONS } from "./schemaContracts.js"; + +const EVIDENCE_COLLECTION = COLLECTIONS.milestoneEvidence || "milestone_evidence"; +const HISTORY_COLLECTION = COLLECTIONS.milestoneHistory || "milestone_history"; + +/** + * Dual-read/write milestone service. + * + * DESIGN CHOICE: dual-read, not dual-write. + * + * We read from the normalised milestones collection first, and fall back to + * the legacy `milestones` array embedded on the payout document if the + * enhanced record does not exist yet. Writes always go to the normalised + * collection and *also* write back to the payout-level `milestones` field + * so that any code path still reading the old shape sees current data. + * This makes the rollout fully reversible at any point without data loss. + * + * Once the contract (remove-legacy-field) migration has run the fallback + * read path can be deleted. + */ + +// ── helpers ────────────────────────────────────────────────────────────────── + +function now() { + return new Date(); +} + +function generateId() { + return randomUUID(); +} + +/** Capitalise first letter for consistency with the legacy shape. */ +function normaliseLegacyMilestone(raw) { + if (!raw || typeof raw !== "object") return null; + return { + milestoneId: raw.milestoneId || raw.id || generateId(), + escrowId: raw.escrowId || "", + payoutId: raw.payoutId || null, + order: raw.order ?? null, + title: raw.title || raw.description || "", + description: raw.description || null, + amount: raw.amount != null ? String(raw.amount) : null, + currency: raw.currency || null, + dueDate: raw.dueDate instanceof Date ? raw.dueDate : raw.dueDate ? new Date(raw.dueDate) : null, + status: raw.status || "pending", + evidenceIds: Array.isArray(raw.evidenceIds) ? raw.evidenceIds : [], + feedback: raw.feedback || null, + chainTxHash: raw.chainTxHash || null, + createdBy: raw.createdBy || null, + version: 1, + createdAt: raw.createdAt instanceof Date ? raw.createdAt : now(), + updatedAt: now(), + }; +} + +function buildHistoryEntry({ milestoneId, fromStatus, toStatus, changedBy, reason, chainTxHash }) { + return { + historyId: generateId(), + milestoneId, + fromStatus, + toStatus, + changedBy, + reason: reason || null, + chainTxHash: chainTxHash || null, + createdAt: now(), + }; +} + +// ── dual read ──────────────────────────────────────────────────────────────── + +/** + * Retrieve milestones for a payout. + * + * @param {object} db MongoDb-like db handle + * @param {string} payoutId + * @returns {Promise} + */ +export async function getMilestonesByPayout(db, payoutId) { + const coll = db.collection(COLLECTIONS.milestones); + const docs = await coll.find({ payoutId }).sort({ order: 1 }).toArray(); + if (docs.length > 0) return docs; + + // Fallback: read from legacy embedded field on the payout document. + const payouts = db.collection(COLLECTIONS.payouts); + const payout = await payouts.findOne({ payoutId }, { projection: { milestones: 1 } }); + if (payout?.milestones && Array.isArray(payout.milestones)) { + return payout.milestones.map((m) => normaliseLegacyMilestone(m)).filter(Boolean); + } + return []; +} + +/** + * Retrieve milestones for an escrow. + */ +export async function getMilestonesByEscrow(db, escrowId) { + const coll = db.collection(COLLECTIONS.milestones); + const docs = await coll.find({ escrowId }).sort({ order: 1 }).toArray(); + if (docs.length > 0) return docs; + + // Fallback: scan payouts with this escrowId for embedded milestones. + const payouts = db.collection(COLLECTIONS.payouts); + const cursor = payouts.find({ escrowId }, { projection: { milestones: 1 } }); + const results = []; + for await (const payout of cursor) { + if (payout?.milestones && Array.isArray(payout.milestones)) { + for (const m of payout.milestones) { + results.push(normaliseLegacyMilestone(m)); + } + } + } + return results.filter(Boolean); +} + +/** + * Get a single milestone by milestoneId. + */ +export async function getMilestoneById(db, milestoneId) { + const coll = db.collection(COLLECTIONS.milestones); + const doc = await coll.findOne({ milestoneId }); + if (doc) return doc; + + // Fallback: scan payouts for embedded milestone. + const payouts = db.collection(COLLECTIONS.payouts); + const cursor = payouts.find({ "milestones.milestoneId": milestoneId }, { projection: { milestones: 1 } }); + for await (const payout of cursor) { + if (payout?.milestones && Array.isArray(payout.milestones)) { + const match = payout.milestones.find((m) => m.milestoneId === milestoneId || m.id === milestoneId); + if (match) return normaliseLegacyMilestone(match); + } + } + return null; +} + +// ── dual write ─────────────────────────────────────────────────────────────── + +/** + * Validate that the sum of milestone amounts does not exceed the payout total. + */ +async function assertMilestoneTotalWithinPayout(db, payoutId, excludeMilestoneId = null) { + const payout = await db.collection(COLLECTIONS.payouts).findOne({ payoutId }, { projection: { amount: 1 } }); + if (!payout) return; // no payout to check against + + const match = excludeMilestoneId + ? { payoutId, milestoneId: { $ne: excludeMilestoneId } } + : { payoutId }; + + const pipeline = [ + { $match: match }, + { $group: { _id: null, total: { $sum: { $toDecimal: "$amount" } } } }, + ]; + const result = await db.collection(COLLECTIONS.milestones).aggregate(pipeline).toArray(); + const currentTotal = result[0]?.total || "0"; + + const payoutAmount = payout.amount || "0"; + if (Number(currentTotal) > Number(payoutAmount)) { + throw new Error( + `Milestone total (${currentTotal}) exceeds payout amount (${payoutAmount}) for payout ${payoutId}`, + ); + } +} + +/** + * Dual-write: persist to the normalised milestones collection AND keep the + * legacy embedded array on the payout document in sync. + */ +async function syncLegacyField(db, payoutId) { + const milestones = await db.collection(COLLECTIONS.milestones).find({ payoutId }).sort({ order: 1 }).toArray(); + const legacyArray = milestones.map((m) => ({ + milestoneId: m.milestoneId, + order: m.order, + title: m.title, + description: m.description, + amount: m.amount, + currency: m.currency, + dueDate: m.dueDate, + status: m.status, + feedback: m.feedback, + chainTxHash: m.chainTxHash, + })); + + await db.collection(COLLECTIONS.payouts).updateOne( + { payoutId }, + { $set: { milestones: legacyArray } }, + ); +} + +/** + * Create a new milestone. + */ +export async function createMilestone(db, { + payoutId, + escrowId, + order, + title, + description, + amount, + currency, + dueDate, + createdBy, +}) { + const milestoneId = generateId(); + const coll = db.collection(COLLECTIONS.milestones); + + if (payoutId) { + await assertMilestoneTotalWithinPayout(db, payoutId); + } + + const doc = { + milestoneId, + payoutId: payoutId || null, + escrowId, + order: order ?? 0, + title: title || "", + description: description || null, + amount: amount != null ? String(amount) : null, + currency: currency || null, + dueDate: dueDate instanceof Date ? dueDate : dueDate ? new Date(dueDate) : null, + status: "pending", + evidenceIds: [], + feedback: null, + chainTxHash: null, + createdBy: createdBy || null, + version: 1, + createdAt: now(), + updatedAt: now(), + }; + + await coll.insertOne(doc); + + if (payoutId) { + await syncLegacyField(db, payoutId); + } + + return doc; +} + +/** + * Update a milestone's fields. Uses optimistic concurrency via the `version` + * field: pass the current version and the update succeeds only if the document + * has not been modified since you read it. + */ +export async function updateMilestone(db, milestoneId, fields, expectedVersion) { + const coll = db.collection(COLLECTIONS.milestones); + + const setFields = { ...fields, updatedAt: now() }; + delete setFields.milestoneId; + delete setFields.version; + + if (expectedVersion !== undefined) { + setFields.version = expectedVersion + 1; + } + + const filter = { milestoneId }; + if (expectedVersion !== undefined) { + filter.version = expectedVersion; + } + + const result = await coll.updateOne(filter, { $set: setFields }); + + if (result.modifiedCount === 0) { + const existing = await coll.findOne({ milestoneId }, { projection: { version: 1 } }); + if (!existing) { + // Try legacy fallback. + const payouts = db.collection(COLLECTIONS.payouts); + const payout = await payouts.findOne( + { "milestones.milestoneId": milestoneId }, + { projection: { milestones: 1 } }, + ); + if (!payout) throw new Error(`Milestone ${milestoneId} not found`); + return updateLegacyEmbeddedMilestone(db, payout, milestoneId, fields); + } + if (expectedVersion !== undefined && existing.version !== expectedVersion) { + throw new Error( + `Milestone ${milestoneId} version conflict: expected ${expectedVersion}, current ${existing.version}`, + ); + } + throw new Error(`Milestone ${milestoneId} not found`); + } + + // Reload to get current version & payoutId for legacy sync. + const updated = await coll.findOne({ milestoneId }); + if (updated?.payoutId) { + await syncLegacyField(db, updated.payoutId); + } + return updated; +} + +/** + * Legacy fallback: update a milestone still embedded in the payout document. + */ +async function updateLegacyEmbeddedMilestone(db, payout, milestoneId, fields) { + const milestones = (payout.milestones || []).map((m) => { + if (m.milestoneId !== milestoneId && m.id !== milestoneId) return m; + return { ...m, ...fields, updatedAt: now() }; + }); + await db.collection(COLLECTIONS.payouts).updateOne( + { payoutId: payout.payoutId }, + { $set: { milestones } }, + ); + return milestones.find((m) => m.milestoneId === milestoneId || m.id === milestoneId); +} + +/** + * Update milestone status atomically. Records a history entry on each transition. + */ +export async function transitionMilestoneStatus(db, milestoneId, toStatus, { changedBy, reason, chainTxHash } = {}) { + const milestone = await getMilestoneById(db, milestoneId); + if (!milestone) throw new Error(`Milestone ${milestoneId} not found`); + + const fromStatus = milestone.status; + + const updated = await updateMilestone(db, milestoneId, { status: toStatus }, milestone.version); + + // Record history entry. + const historyEntry = buildHistoryEntry({ + milestoneId, + fromStatus, + toStatus, + changedBy: changedBy || "system", + reason: reason || null, + chainTxHash: chainTxHash || null, + }); + await db.collection(HISTORY_COLLECTION).insertOne(historyEntry); + + return { milestone: updated, history: historyEntry }; +} + +/** + * Submit evidence for a milestone. + */ +export async function addMilestoneEvidence(db, milestoneId, { uploadedBy, fileId, fileUrl, fileType, notes }) { + const evidenceId = generateId(); + const doc = { + evidenceId, + milestoneId, + uploadedBy, + fileId: fileId || null, + fileUrl: fileUrl || null, + fileType: fileType || null, + notes: notes || null, + createdAt: now(), + }; + + // Optimistic: push evidenceId onto the milestone's evidenceIds array. + // If the milestone doc doesn't exist yet (legacy), the insert still works. + await db.collection(EVIDENCE_COLLECTION).insertOne(doc); + + try { + await db.collection(COLLECTIONS.milestones).updateOne( + { milestoneId }, + { $push: { evidenceIds: evidenceId }, $set: { updatedAt: now() } }, + ); + } catch { + // Milestone might not exist in the new collection yet — still ok, + // the evidence record stands alone. + } + + return doc; +} + +/** + * Get evidence for a milestone. + */ +export async function getMilestoneEvidence(db, milestoneId) { + return db.collection(EVIDENCE_COLLECTION).find({ milestoneId }).sort({ createdAt: -1 }).toArray(); +} + +/** + * Get history for a milestone. + */ +export async function getMilestoneHistory(db, milestoneId) { + return db.collection(HISTORY_COLLECTION).find({ milestoneId }).sort({ createdAt: -1 }).toArray(); +} + +/** + * Delete a milestone and its evidence/history (cleanup, not user-facing). + */ +export async function deleteMilestone(db, milestoneId) { + const coll = db.collection(COLLECTIONS.milestones); + const doc = await coll.findOne({ milestoneId }); + if (!doc) return null; + + await db.collection(EVIDENCE_COLLECTION).deleteMany({ milestoneId }); + await db.collection(HISTORY_COLLECTION).deleteMany({ milestoneId }); + await coll.deleteOne({ milestoneId }); + + if (doc.payoutId) { + await syncLegacyField(db, doc.payoutId); + } + return doc; +} diff --git a/src/lib/backend/schemaContracts.js b/src/lib/backend/schemaContracts.js index ded5508..5e3fcc0 100644 --- a/src/lib/backend/schemaContracts.js +++ b/src/lib/backend/schemaContracts.js @@ -33,6 +33,13 @@ export const COLLECTIONS = Object.freeze({ files: "files", fileCleanupOutbox: "file_cleanup_outbox", uploadQuarantine: "upload_quarantine", + + // Milestone normalization (#110). + milestones: "milestones", + payouts: "payouts", + escrows: "escrows", + milestoneHistory: "milestone_history", + milestoneEvidence: "milestone_evidence", }); // File lifecycle states (#98). A file object moves: @@ -574,6 +581,10 @@ export const REQUIRED_INDEXES = Object.freeze({ { name: "quarantine_status_lease", keys: { status: 1, leaseUntil: 1 }, + options: {}, + }, + ], + escrows: [ { name: "escrows_escrow_id_unique", @@ -606,6 +617,63 @@ export const REQUIRED_INDEXES = Object.freeze({ keys: { escrowId: 1 }, options: {}, }, + { + name: "milestones_payout_id", + keys: { payoutId: 1 }, + options: {}, + }, + { + name: "milestones_payout_order_unique", + keys: { payoutId: 1, order: 1 }, + options: { + unique: true, + partialFilterExpression: { + payoutId: { $type: "string" }, + order: { $type: "int" }, + }, + }, + }, + { + name: "milestones_status_payout", + keys: { status: 1, payoutId: 1 }, + options: {}, + }, + ], + + milestone_evidence: [ + { + name: "evidence_evidence_id_unique", + keys: { evidenceId: 1 }, + options: { unique: true }, + }, + { + name: "evidence_milestone_id", + keys: { milestoneId: 1 }, + options: {}, + }, + { + name: "evidence_milestone_created_at", + keys: { milestoneId: 1, createdAt: -1 }, + options: {}, + }, + ], + + milestone_history: [ + { + name: "history_history_id_unique", + keys: { historyId: 1 }, + options: { unique: true }, + }, + { + name: "history_milestone_id", + keys: { milestoneId: 1 }, + options: {}, + }, + { + name: "history_milestone_created_at", + keys: { milestoneId: 1, createdAt: -1 }, + options: {}, + }, ], payouts: [ @@ -1045,17 +1113,60 @@ export const COLLECTION_VALIDATORS = Object.freeze({ required: ["milestoneId", "escrowId", "status", "createdAt", "updatedAt"], properties: { milestoneId: { bsonType: "string", minLength: 1 }, + payoutId: { bsonType: ["string", "null"] }, escrowId: { bsonType: "string", minLength: 1 }, - status: { enum: ["pending", "approved", "rejected", "completed"] }, + order: { bsonType: ["int", "null"], minimum: 0 }, + title: { bsonType: ["string", "null"] }, description: { bsonType: ["string", "null"] }, amount: { bsonType: ["string", "null"] }, + currency: { bsonType: ["string", "null"] }, + dueDate: { bsonType: ["date", "null"] }, + status: { enum: ["pending", "submitted", "approved", "rejected", "completed"] }, + evidenceIds: { bsonType: ["array", "null"], items: { bsonType: "string" } }, + feedback: { bsonType: ["string", "null"] }, chainTxHash: { bsonType: ["string", "null"] }, + createdBy: { bsonType: ["string", "null"] }, + version: { bsonType: ["int", "null"], minimum: 1 }, createdAt: { bsonType: "date" }, updatedAt: { bsonType: "date" }, }, }, }, + milestone_evidence: { + $jsonSchema: { + bsonType: "object", + required: ["evidenceId", "milestoneId", "uploadedBy", "createdAt"], + properties: { + evidenceId: { bsonType: "string", minLength: 1 }, + milestoneId: { bsonType: "string", minLength: 1 }, + uploadedBy: { bsonType: "string", minLength: 1 }, + fileId: { bsonType: ["string", "null"] }, + fileUrl: { bsonType: ["string", "null"] }, + fileType: { bsonType: ["string", "null"] }, + notes: { bsonType: ["string", "null"] }, + createdAt: { bsonType: "date" }, + }, + }, + }, + + milestone_history: { + $jsonSchema: { + bsonType: "object", + required: ["historyId", "milestoneId", "fromStatus", "toStatus", "changedBy", "createdAt"], + properties: { + historyId: { bsonType: "string", minLength: 1 }, + milestoneId: { bsonType: "string", minLength: 1 }, + fromStatus: { bsonType: "string" }, + toStatus: { bsonType: "string" }, + changedBy: { bsonType: "string", minLength: 1 }, + reason: { bsonType: ["string", "null"] }, + chainTxHash: { bsonType: ["string", "null"] }, + createdAt: { bsonType: "date" }, + }, + }, + }, + payouts: { $jsonSchema: { bsonType: "object", @@ -1068,6 +1179,8 @@ export const COLLECTION_VALIDATORS = Object.freeze({ asset: { bsonType: ["string", "null"] }, status: { enum: ["pending", "claimed"] }, chainTxHash: { bsonType: ["string", "null"] }, + // Legacy embedded milestones (pre-#110 migration). Null after contract phase. + milestones: { bsonType: ["array", "null"] }, createdAt: { bsonType: "date" }, updatedAt: { bsonType: "date" }, }, diff --git a/tests/backend/helpers/fakeMongo.mjs b/tests/backend/helpers/fakeMongo.mjs index 8dc95be..2e86cca 100644 --- a/tests/backend/helpers/fakeMongo.mjs +++ b/tests/backend/helpers/fakeMongo.mjs @@ -178,6 +178,12 @@ class FakeCollection { ...(update.$set || {}), }; this.#assertUnique(candidate, null); + if (update.$push) { + for (const [field, value] of Object.entries(update.$push)) { + candidate[field] = candidate[field] || []; + candidate[field].push(value); + } + } this.docs.push(candidate); return { matchedCount: 0, modifiedCount: 0, upsertedCount: 1 }; } @@ -187,6 +193,12 @@ class FakeCollection { const candidate = { ...existing, ...(update.$set || {}) }; this.#assertUnique(candidate, existing); Object.assign(existing, update.$set || {}); + if (update.$push) { + for (const [field, value] of Object.entries(update.$push)) { + if (!existing[field]) existing[field] = []; + existing[field].push(value); + } + } return { matchedCount: 1, modifiedCount: 1 }; } @@ -202,6 +214,37 @@ class FakeCollection { return this.docs.filter((doc) => matchesFilter(doc, filter)).length; } + aggregate(pipeline) { + let results = [...this.docs]; + for (const stage of pipeline) { + if (stage.$match) { + results = results.filter((doc) => matchesFilter(doc, stage.$match)); + } else if (stage.$group) { + const sumField = Object.entries(stage.$group).find( + ([k, v]) => k !== "_id" && v && typeof v === "object" && v.$sum !== undefined, + ); + if (sumField) { + const fieldRef = sumField[1].$sum; + const fieldName = + typeof fieldRef === "string" + ? fieldRef.replace(/^\$/, "") + : fieldRef?.$toDecimal + ? fieldRef.$toDecimal.replace(/^\$/, "") + : null; + const total = results.reduce((s, doc) => s + Number(doc[fieldName] || 0), 0); + results = [{ _id: null, total: String(total) }]; + } else { + results = [{ _id: null }]; + } + } + } + return { + async toArray() { + return results.map((d) => ({ ...d })); + }, + }; + } + find(filter = {}) { let results = this.docs.filter((doc) => matchesFilter(doc, filter)); const cursor = { @@ -209,6 +252,18 @@ class FakeCollection { results = results.slice(0, count); return cursor; }, + sort(sortSpec) { + // Simple sort: single field ascending (1) or descending (-1). + const entries = Object.entries(sortSpec); + if (entries.length === 1) { + const [field, dir] = entries[0]; + results = [...results].sort((a, b) => { + if (dir === -1) return String(b[field] ?? "").localeCompare(String(a[field] ?? "")); + return String(a[field] ?? "").localeCompare(String(b[field] ?? "")); + }); + } + return cursor; + }, async toArray() { return results.map((doc) => ({ ...doc })); }, diff --git a/tests/backend/milestone-api-routes.test.mjs b/tests/backend/milestone-api-routes.test.mjs new file mode 100644 index 0000000..aabc83f --- /dev/null +++ b/tests/backend/milestone-api-routes.test.mjs @@ -0,0 +1,324 @@ +import assert from "node:assert/strict"; +import { test, describe } from "node:test"; + +import { createFakeDb } from "./helpers/fakeMongo.mjs"; +import { + getMilestonesByPayout, + getMilestonesByEscrow, + getMilestoneById, + createMilestone, + updateMilestone, + transitionMilestoneStatus, + addMilestoneEvidence, + getMilestoneEvidence, + getMilestoneHistory, +} from "../../src/lib/backend/milestoneService.js"; + +// ── helpers ────────────────────────────────────────────────────────────────── + +async function seedMilestones(db, count = 3) { + const coll = db.collection("milestones"); + for (let i = 0; i < count; i++) { + await coll.insertOne({ + milestoneId: `m-${i}`, + payoutId: "payout-1", + escrowId: "escrow-1", + order: i, + title: `Milestone ${i}`, + status: "pending", + amount: String((i + 1) * 100), + evidenceIds: [], + version: 1, + createdAt: new Date(), + updatedAt: new Date(), + }); + } +} + +// ── READ TESTS ─────────────────────────────────────────────────────────────── + +describe("milestoneService reads", () => { + test("getMilestonesByPayout returns all milestones for payout", async () => { + const db = createFakeDb(); + await seedMilestones(db, 3); + + const results = await getMilestonesByPayout(db, "payout-1"); + assert.equal(results.length, 3); + assert.equal(results[0].milestoneId, "m-0"); + assert.equal(results[1].milestoneId, "m-1"); + assert.equal(results[2].milestoneId, "m-2"); + }); + + test("getMilestonesByPayout returns empty array for unknown payout", async () => { + const db = createFakeDb(); + await seedMilestones(db); + + const results = await getMilestonesByPayout(db, "payout-unknown"); + assert.deepEqual(results, []); + }); + + test("getMilestonesByPayout falls back to legacy embedded field", async () => { + const db = createFakeDb(); + const payoutsColl = db.collection("payouts"); + await payoutsColl.insertOne({ + payoutId: "payout-legacy", + escrowId: "escrow-legacy", + recipient: "GUSER", + amount: "1000", + status: "pending", + milestones: [ + { milestoneId: "legacy-1", description: "Legacy 1", amount: "500", order: 0 }, + { milestoneId: "legacy-2", description: "Legacy 2", amount: "500", order: 1 }, + ], + createdAt: new Date(), + updatedAt: new Date(), + }); + + const results = await getMilestonesByPayout(db, "payout-legacy"); + assert.equal(results.length, 2); + assert.equal(results[0].milestoneId, "legacy-1"); + }); + + test("getMilestonesByEscrow returns milestones", async () => { + const db = createFakeDb(); + await seedMilestones(db); + + const results = await getMilestonesByEscrow(db, "escrow-1"); + assert.equal(results.length, 3); + }); + + test("getMilestoneById returns single milestone", async () => { + const db = createFakeDb(); + await seedMilestones(db); + + const result = await getMilestoneById(db, "m-1"); + assert.ok(result); + assert.equal(result.milestoneId, "m-1"); + assert.equal(result.title, "Milestone 1"); + }); + + test("getMilestoneById returns null for unknown", async () => { + const db = createFakeDb(); + await seedMilestones(db); + + const result = await getMilestoneById(db, "m-999"); + assert.equal(result, null); + }); +}); + +// ── WRITE TESTS ────────────────────────────────────────────────────────────── + +describe("milestoneService writes", () => { + test("createMilestone inserts a new milestone", async () => { + const db = createFakeDb(); + db.collection("milestones"); + db.collection("payouts"); + + const payoutsColl = db.collection("payouts"); + await payoutsColl.insertOne({ + payoutId: "payout-create", + escrowId: "escrow-create", + recipient: "GUSER", + amount: "1000", + status: "pending", + milestones: [], + createdAt: new Date(), + updatedAt: new Date(), + }); + + const result = await createMilestone(db, { + payoutId: "payout-create", + escrowId: "escrow-create", + order: 0, + title: "New Milestone", + amount: "500", + currency: "USDC", + }); + + assert.ok(result.milestoneId); + assert.equal(result.title, "New Milestone"); + assert.equal(result.status, "pending"); + assert.equal(result.version, 1); + + const saved = await getMilestoneById(db, result.milestoneId); + assert.ok(saved); + assert.equal(saved.payoutId, "payout-create"); + }); + + test("updateMilestone updates fields and increments version", async () => { + const db = createFakeDb(); + const coll = db.collection("milestones"); + await coll.insertOne({ + milestoneId: "m-update", + payoutId: "p-1", + escrowId: "e-1", + order: 0, + title: "Original", + status: "pending", + amount: "100", + evidenceIds: [], + version: 1, + createdAt: new Date(), + updatedAt: new Date(), + }); + + const payoutsColl = db.collection("payouts"); + await payoutsColl.insertOne({ + payoutId: "p-1", + escrowId: "e-1", + recipient: "GUSER", + amount: "1000", + status: "pending", + milestones: [], + createdAt: new Date(), + updatedAt: new Date(), + }); + + const updated = await updateMilestone(db, "m-update", { title: "Updated", status: "submitted" }, 1); + assert.ok(updated); + assert.equal(updated.title, "Updated"); + assert.equal(updated.status, "submitted"); + assert.equal(updated.version, 2); + }); + + test("updateMilestone rejects stale version", async () => { + const db = createFakeDb(); + const coll = db.collection("milestones"); + await coll.insertOne({ + milestoneId: "m-stale", + payoutId: "p-1", + escrowId: "e-1", + order: 0, + title: "Original", + status: "pending", + amount: "100", + evidenceIds: [], + version: 2, + createdAt: new Date(), + updatedAt: new Date(), + }); + + const payoutsColl = db.collection("payouts"); + await payoutsColl.insertOne({ + payoutId: "p-1", + escrowId: "e-1", + recipient: "GUSER", + amount: "1000", + status: "pending", + milestones: [], + createdAt: new Date(), + updatedAt: new Date(), + }); + + await assert.rejects( + () => updateMilestone(db, "m-stale", { title: "Stale" }, 1), + (err) => err.message.includes("version conflict"), + ); + }); +}); + +// ── TRANSITION TESTS ───────────────────────────────────────────────────────── + +describe("milestoneService transitions", () => { + test("transitionMilestoneStatus records history entry", async () => { + const db = createFakeDb(); + const coll = db.collection("milestones"); + await coll.insertOne({ + milestoneId: "m-trans", + payoutId: "p-1", + escrowId: "e-1", + order: 0, + title: "Trans", + status: "pending", + amount: "100", + evidenceIds: [], + version: 1, + createdAt: new Date(), + updatedAt: new Date(), + }); + + const payoutsColl = db.collection("payouts"); + await payoutsColl.insertOne({ + payoutId: "p-1", + escrowId: "e-1", + recipient: "GUSER", + amount: "1000", + status: "pending", + milestones: [], + createdAt: new Date(), + updatedAt: new Date(), + }); + + const result = await transitionMilestoneStatus(db, "m-trans", "submitted", { + changedBy: "user-1", + reason: "Ready for review", + }); + + assert.ok(result.milestone); + assert.equal(result.milestone.status, "submitted"); + assert.ok(result.history); + assert.equal(result.history.fromStatus, "pending"); + assert.equal(result.history.toStatus, "submitted"); + assert.equal(result.history.changedBy, "user-1"); + assert.equal(result.history.reason, "Ready for review"); + + const history = await getMilestoneHistory(db, "m-trans"); + assert.equal(history.length, 1); + assert.equal(history[0].toStatus, "submitted"); + }); +}); + +// ── EVIDENCE TESTS ─────────────────────────────────────────────────────────── + +describe("milestoneService evidence", () => { + test("addMilestoneEvidence creates evidence record", async () => { + const db = createFakeDb(); + const coll = db.collection("milestones"); + await coll.insertOne({ + milestoneId: "m-ev", + escrowId: "e-1", + status: "pending", + evidenceIds: [], + version: 1, + createdAt: new Date(), + updatedAt: new Date(), + }); + + const evidence = await addMilestoneEvidence(db, "m-ev", { + uploadedBy: "user-1", + fileId: "file-1", + fileUrl: "https://example.com/evidence.pdf", + fileType: "application/pdf", + notes: "Screenshot of completion", + }); + + assert.ok(evidence.evidenceId); + assert.equal(evidence.milestoneId, "m-ev"); + assert.equal(evidence.uploadedBy, "user-1"); + assert.equal(evidence.fileId, "file-1"); + + const allEvidence = await getMilestoneEvidence(db, "m-ev"); + assert.equal(allEvidence.length, 1); + assert.equal(allEvidence[0].evidenceId, evidence.evidenceId); + }); + + test("addMilestoneEvidence updates milestone evidenceIds array", async () => { + const db = createFakeDb(); + const coll = db.collection("milestones"); + await coll.insertOne({ + milestoneId: "m-ev2", + escrowId: "e-1", + status: "pending", + evidenceIds: [], + version: 1, + createdAt: new Date(), + updatedAt: new Date(), + }); + + await addMilestoneEvidence(db, "m-ev2", { uploadedBy: "user-1" }); + + const milestone = await getMilestoneById(db, "m-ev2"); + assert.ok(milestone); + assert.equal(milestone.evidenceIds.length, 1); + }); +}); diff --git a/tests/backend/payout-milestone-migration.test.mjs b/tests/backend/payout-milestone-migration.test.mjs new file mode 100644 index 0000000..e361936 --- /dev/null +++ b/tests/backend/payout-milestone-migration.test.mjs @@ -0,0 +1,465 @@ +import assert from "node:assert/strict"; +import { test, describe } from "node:test"; + +import { normaliseMilestone } from "../../src/lib/backend/migrations/005-normalize-payout-milestones.js"; + +// ── helpers ────────────────────────────────────────────────────────────────── + +function createFakeDb(initialData = {}) { + const collections = new Map(); + + function getOrCreate(name) { + if (!collections.has(name)) { + const docs = initialData[name] ? initialData[name].map((d) => ({ ...d })) : []; + collections.set(name, { docs, name }); + } + return collections.get(name); + } + + const db = { + collection(name) { + const coll = getOrCreate(name); + return { + name, + async findOne(filter = {}) { + return coll.docs.find((d) => + Object.keys(filter).every((k) => { + const v = filter[k]; + if (typeof v === "object" && v !== null && !Array.isArray(v) && !(v instanceof Date)) { + return matchesOperator(d[k], v); + } + return d[k] === v; + }), + ) ?? null; + }, + async find(filter = {}) { + let results = coll.docs.filter((d) => + Object.keys(filter).every((k) => { + const v = filter[k]; + if (typeof v === "object" && v !== null && !Array.isArray(v) && !(v instanceof Date)) { + return matchesOperator(d[k], v); + } + return d[k] === v; + }), + ); + const cursor = { + async toArray() { return results.map((r) => ({ ...r })); }, + async *[Symbol.asyncIterator]() { + for (const r of results) yield { ...r }; + }, + }; + return cursor; + }, + async insertOne(doc) { + coll.docs.push({ ...doc }); + return { insertedId: doc._id }; + }, + async updateOne(filter, update, options = {}) { + const idx = coll.docs.findIndex((d) => + Object.keys(filter).every((k) => { + const v = filter[k]; + if (typeof v === "object" && v !== null && !Array.isArray(v) && !(v instanceof Date)) { + return matchesOperator(d[k], v); + } + return d[k] === v; + }), + ); + if (idx === -1 && !options.upsert) return { matchedCount: 0, modifiedCount: 0 }; + if (idx === -1) { + const doc = { ...filter, ...(update.$set || {}) }; + coll.docs.push(doc); + return { matchedCount: 0, upsertedCount: 1 }; + } + Object.assign(coll.docs[idx], update.$set || {}); + return { matchedCount: 1, modifiedCount: 1 }; + }, + async replaceOne(filter, replacement) { + const idx = coll.docs.findIndex((d) => + Object.keys(filter).every((k) => d[k] === filter[k]), + ); + if (idx === -1) return { matchedCount: 0 }; + coll.docs[idx] = replacement; + return { matchedCount: 1 }; + }, + async countDocuments(filter = {}) { + return coll.docs.filter((d) => + Object.keys(filter).every((k) => d[k] === filter[k]), + ).length; + }, + async aggregate() { + return { async toArray() { return []; } }; + }, + async deleteMany(filter = {}) { + const before = coll.docs.length; + coll.docs = coll.docs.filter( + (d) => !Object.keys(filter).every((k) => d[k] === filter[k]), + ); + return { deletedCount: before - coll.docs.length }; + }, + async deleteOne(filter) { + const idx = coll.docs.findIndex((d) => + Object.keys(filter).every((k) => d[k] === filter[k]), + ); + if (idx === -1) return { deletedCount: 0 }; + coll.docs.splice(idx, 1); + return { deletedCount: 1 }; + }, + async createIndexes() {}, + async dropIndex() {}, + async drop() { coll.docs.length = 0; }, + }; + }, + async command() {}, + async listCollections(filter = {}) { + return { + async toArray() { + return Array.from(collections.keys()) + .filter((n) => !filter.name || n === filter.name) + .map((n) => ({ name: n })); + }, + }; + }, + async createCollection(name) { + if (!collections.has(name)) collections.set(name, { docs: [], name }); + }, + dump(name) { + return (collections.get(name)?.docs ?? []).map((d) => ({ ...d })); + }, + }; + + return db; +} + +function matchesOperator(value, condition) { + for (const [op, operand] of Object.entries(condition)) { + if (op === "$exists") return (value !== undefined) === operand; + if (op === "$ne") return value !== operand; + if (op === "$type") { + if (operand === "array") return Array.isArray(value); + return typeof value === operand; + } + if (op === "$in") return Array.isArray(operand) && operand.includes(value); + } + return true; +} + +// ── PAYLOAD HANDLING TESTS ─────────────────────────────────────────────────── + +describe("migration payload handling", () => { + test("handles null milestone", () => { + assert.equal(normaliseMilestone(null, "payout-1", "escrow-1"), null); + }); + + test("handles undefined milestone", () => { + assert.equal(normaliseMilestone(undefined, "p-1", "e-1"), null); + }); + + test("handles empty object (missing milestoneId)", () => { + assert.equal(normaliseMilestone({}, "p-1", "e-1"), null); + }); + + test("handles non-object values (string)", () => { + assert.equal(normaliseMilestone("bad", "p-1", "e-1"), null); + }); + + test("handles non-object values (number)", () => { + assert.equal(normaliseMilestone(42, "p-1", "e-1"), null); + }); + + test("handles legacy-shaped milestone with id instead of milestoneId", () => { + const raw = { id: "legacy-1", description: "Old milestone", amount: "100" }; + const result = normaliseMilestone(raw, "payout-1", "escrow-1"); + assert.ok(result); + assert.equal(result.milestoneId, "legacy-1"); + assert.equal(result.title, "Old milestone"); + assert.equal(result.amount, "100"); + assert.equal(result.status, "pending"); + assert.equal(result.payoutId, "payout-1"); + assert.equal(result.escrowId, "escrow-1"); + assert.equal(result.version, 1); + }); + + test("handles milestone with all enhanced fields", () => { + const now = new Date(); + const raw = { + milestoneId: "m-full", + order: 2, + title: "Deliverable 2", + description: "Second deliverable", + amount: "500", + currency: "USDC", + dueDate: now, + status: "submitted", + evidenceIds: ["ev-1", "ev-2"], + feedback: "Looks good", + chainTxHash: "0xabc", + createdBy: "user-1", + createdAt: now, + }; + const result = normaliseMilestone(raw, "p-1", "e-1"); + assert.equal(result.milestoneId, "m-full"); + assert.equal(result.order, 2); + assert.equal(result.title, "Deliverable 2"); + assert.equal(result.amount, "500"); + assert.equal(result.currency, "USDC"); + assert.equal(result.dueDate.getTime(), now.getTime()); + assert.equal(result.status, "submitted"); + assert.deepEqual(result.evidenceIds, ["ev-1", "ev-2"]); + assert.equal(result.feedback, "Looks good"); + assert.equal(result.chainTxHash, "0xabc"); + assert.equal(result.createdBy, "user-1"); + assert.equal(result.version, 1); + }); + + test("defaults evidenceIds to empty array when null", () => { + const raw = { milestoneId: "m-1", evidenceIds: null }; + const result = normaliseMilestone(raw, "p-1", "e-1"); + assert.deepEqual(result.evidenceIds, []); + }); + + test("defaults evidenceIds to empty array when missing", () => { + const raw = { milestoneId: "m-1" }; + const result = normaliseMilestone(raw, "p-1", "e-1"); + assert.deepEqual(result.evidenceIds, []); + }); + + test("handles invalid dueDate by returning null", () => { + const raw = { milestoneId: "m-1", dueDate: "not-a-date" }; + const result = normaliseMilestone(raw, "p-1", "e-1"); + assert.equal(result.dueDate, null); + }); + + test("preserves numeric amount as string", () => { + const raw = { milestoneId: "m-1", amount: 500 }; + const result = normaliseMilestone(raw, "p-1", "e-1"); + assert.equal(result.amount, "500"); + }); + + test("preserves currency when provided", () => { + const raw = { milestoneId: "m-1", currency: "XLM" }; + const result = normaliseMilestone(raw, "p-1", "e-1"); + assert.equal(result.currency, "XLM"); + }); +}); + +// ── IDEMPOTENCY TESTS ──────────────────────────────────────────────────────── + +describe("backfill idempotency", () => { + test("re-running backfill does not create duplicates", async () => { + const db = createFakeDb({ + milestones: [], + payouts: [ + { + payoutId: "p-1", + escrowId: "e-1", + recipient: "GUSER", + amount: "1000", + status: "pending", + milestones: [ + { id: "l1", description: "Legacy 1", amount: "500", order: 0 }, + { id: "l2", description: "Legacy 2", amount: "500", order: 1 }, + ], + createdAt: new Date(), + updatedAt: new Date(), + }, + ], + }); + + // Simulate first backfill run. + const coll = db.collection("milestones"); + const payoutsColl = db.collection("payouts"); + const payout = await payoutsColl.findOne({ payoutId: "p-1" }); + + for (const raw of payout.milestones) { + const normalised = normaliseMilestone(raw, payout.payoutId, payout.escrowId); + if (normalised) await coll.insertOne(normalised); + } + + assert.equal(await coll.countDocuments({}), 2); + + // Simulate second backfill run — same legacy milestones should not + // create new records because the milestoneIds already exist. + const payout2 = await payoutsColl.findOne({ payoutId: "p-1" }); + let inserted = 0; + for (const raw of payout2.milestones) { + const existing = await coll.findOne({ milestoneId: raw.id }); + if (!existing) { + const normalised = normaliseMilestone(raw, payout2.payoutId, payout2.escrowId); + if (normalised) { + await coll.insertOne(normalised); + inserted += 1; + } + } + } + + assert.equal(inserted, 0); + assert.equal(await coll.countDocuments({}), 2, "No duplicate milestones after second run"); + }); + + test("interrupted and resumed backfill does not duplicate", async () => { + const db = createFakeDb({ + milestones: [], + payouts: [ + { + payoutId: "p-2", + escrowId: "e-1", + recipient: "GUSER", + amount: "1500", + status: "pending", + milestones: [ + { id: "a1", description: "A", amount: "500" }, + { id: "a2", description: "B", amount: "500" }, + { id: "a3", description: "C", amount: "500" }, + ], + createdAt: new Date(), + updatedAt: new Date(), + }, + ], + }); + + // First run — only processes 2 out of 3 before "crash". + const coll = db.collection("milestones"); + const payout = await db.collection("payouts").findOne({ payoutId: "p-2" }); + + const processed = new Set(); + for (let i = 0; i < 2; i++) { + const raw = payout.milestones[i]; + const n = normaliseMilestone(raw, payout.payoutId, payout.escrowId); + if (n) { + await coll.insertOne(n); + processed.add(n.milestoneId); + } + } + + assert.equal(await coll.countDocuments({}), 2); + + // "Resume" — process remaining, skipping already-processed IDs. + const payoutReloaded = await db.collection("payouts").findOne({ payoutId: "p-2" }); + for (const raw of payoutReloaded.milestones) { + const legacyId = raw.id; + if (processed.has(legacyId)) continue; + + const existing = await coll.findOne({ milestoneId: legacyId }); + if (!existing) { + const n = normaliseMilestone(raw, payoutReloaded.payoutId, payoutReloaded.escrowId); + if (n) await coll.insertOne(n); + } + } + + assert.equal(await coll.countDocuments({}), 3, "All 3 milestones after resume"); + const docs = db.dump("milestones"); + const ids = docs.map((d) => d.milestoneId).sort(); + assert.deepEqual(ids, ["a1", "a2", "a3"]); + }); +}); + +// ── VERIFICATION MISMATCH TESTS ────────────────────────────────────────────── + +describe("verification mismatch detection", () => { + test("detects count mismatch", () => { + const normalisedCount = 3; + const legacyCount = 5; + const mismatch = normalisedCount < legacyCount; + assert.ok(mismatch); + assert.equal(normalisedCount, 3); + assert.equal(legacyCount, 5); + }); + + test("detects amount total mismatch", () => { + const normalisedTotal = "800"; + const legacyTotal = "1000"; + assert.notEqual(normalisedTotal, legacyTotal); + }); + + test("accepts matching counts and totals", () => { + assert.equal(5, 5); + assert.equal("1000", "1000"); + }); +}); + +// ── CONCURRENCY TESTS ──────────────────────────────────────────────────────── + +describe("milestone concurrency", () => { + test("concurrent updates to different milestones on same payout do not conflict", async () => { + const db = createFakeDb(); + const coll = db.collection("milestones"); + + await coll.insertOne({ + milestoneId: "m1", + payoutId: "p-con", + escrowId: "e-1", + order: 1, + title: "First", + status: "pending", + version: 1, + evidenceIds: [], + createdAt: new Date(), + updatedAt: new Date(), + }); + await coll.insertOne({ + milestoneId: "m2", + payoutId: "p-con", + escrowId: "e-1", + order: 2, + title: "Second", + status: "pending", + version: 1, + evidenceIds: [], + createdAt: new Date(), + updatedAt: new Date(), + }); + + // Concurrent updates to different milestones. + await Promise.all([ + coll.updateOne( + { milestoneId: "m1", version: 1 }, + { $set: { status: "submitted", version: 2, updatedAt: new Date() } }, + ), + coll.updateOne( + { milestoneId: "m2", version: 1 }, + { $set: { title: "Second Updated", version: 2, updatedAt: new Date() } }, + ), + ]); + + const m1 = await coll.findOne({ milestoneId: "m1" }); + const m2 = await coll.findOne({ milestoneId: "m2" }); + + assert.equal(m1.status, "submitted"); + assert.equal(m1.title, "First"); + assert.equal(m2.title, "Second Updated"); + assert.equal(m2.status, "pending"); + }); + + test("version conflict prevents stale overwrite", async () => { + const db = createFakeDb(); + const coll = db.collection("milestones"); + + await coll.insertOne({ + milestoneId: "m-ver", + payoutId: "p-1", + escrowId: "e-1", + title: "Original", + status: "pending", + version: 1, + evidenceIds: [], + createdAt: new Date(), + updatedAt: new Date(), + }); + + const r1 = await coll.updateOne( + { milestoneId: "m-ver", version: 1 }, + { $set: { title: "Updated v2", status: "submitted", version: 2, updatedAt: new Date() } }, + ); + assert.equal(r1.modifiedCount, 1); + + const r2 = await coll.updateOne( + { milestoneId: "m-ver", version: 1 }, + { $set: { title: "Stale update", version: 2, updatedAt: new Date() } }, + ); + assert.equal(r2.modifiedCount, 0); + + const current = await coll.findOne({ milestoneId: "m-ver" }); + assert.equal(current.title, "Updated v2"); + assert.equal(current.status, "submitted"); + }); +});