Skip to content

Account-scoped data portability: export/import (not backup) #46

Description

@sliptonic

Problem Statement

My tool data lives on one Loobric instance, but instances are not forever: I might move from the hosted sandbox to a self-hosted box in the shop, replace a machine that hosts my solo server, consolidate two instances, or simply want my data in my own hands. Today the only whole-data operation is server backup, which is admin-gated and server-shaped — it is the operator's disaster-recovery tool, not the user's. As an ordinary account holder I have no way to take my data — records, provenance, labels, usage history, media — out of one Loobric and put it into another where I may have a different email, a different password, and no admin rights. For an open-core product whose pitch includes "no lock-in," that escape hatch needs to be real, owner-operated, and complete.

Solution

Account-scoped export and import: any signed-in user can download a single portable bundle containing everything their account owns, and upload that bundle into an empty account on any Loobric instance. The bundle preserves the data exactly — every canonical field keeps its provenance tag, derived usage totals keep their ledger, labels keep their codes, media keeps its bytes — while ownership is re-attributed to the importing account. Export is one click on the account page (or one CLI/API call); import is the mirror. Backup remains the admin's tool; portability is the user's.

User Stories

  1. As a Loobric user, I want to export all my tool data as a single downloadable file, so that my data is in my own hands regardless of where the server runs.
  2. As a self-hosting user, I want to import that file into a fresh account on my own server, so that I can leave the hosted instance without losing anything.
  3. As a hosted user, I want to import an export from my old self-hosted box, so that migration works in both directions.
  4. As a user, I want every canonical field to arrive with its original provenance source intact, so that an observed measurement is still observed and an assertion is still attributed after the move.
  5. As a user, I want my derived usage totals to arrive with their underlying contribution ledger, so that the decomposition ("37.4 = 25.3 + 12.1") still works on the new instance.
  6. As a user, I want my labels to arrive with their codes, so that already-printed stickers can be re-pointed at the new instance rather than re-cataloged from scratch.
  7. As a user, I want to be told clearly at import time that printed sticker URLs still point at the old host, so that I know reprinting (or a redirect at the old host) is on me.
  8. As a user, I want my media (photos, STEP models, drawings) to travel inside the bundle, so that records don't arrive with dangling references.
  9. As a user, I want my machines, tool-table entries, bindings, setups (including setup history), tool sets, and open binding proposals to all come along, so that the shop state is complete, not just the tool list.
  10. As a user, I want the import to be re-attributed to my account on the new instance, so that a different email/identity on the target is not an obstacle.
  11. As a user, I want import to refuse politely if my target account already has tool data, so that I can't accidentally scramble two datasets together.
  12. As a user, I want a summary report after import (counts per record type, anything skipped and why), so that I can verify completeness at a glance.
  13. As a user, I want export/import buttons on my account page, so that portability doesn't require the CLI.
  14. As a CLI user, I want loobric verbs for export and import, so that migration can be scripted.
  15. As a user, I want export to require only my normal sign-in (no admin), so that portability is a right of the account, not a favor from the operator.
  16. As a security-conscious user, I want my password hash and API keys excluded from the bundle, so that the file on my laptop is tool data, not credentials.
  17. As a user importing an old bundle, I want a clear version check with an honest error, so that a bundle from a much newer/older server fails loudly instead of importing garbage.
  18. As a user whose bundle contains a label code that already exists on the target instance, I want the collision reported and resolved deterministically, so that no label silently points at the wrong tool.
  19. As an instance operator, I want import to write through a privileged account-scoped path rather than the public doors, so that provenance is preserved verbatim and nothing is re-stamped as an importer's assertion.
  20. As an instance operator, I want the target server's own audit log to record the import as one attributed act, so that "this data arrived by import on this date" is server truth.
  21. As a user, I want the old server's audit log left behind (not forged into the new server's), so that each server's testimony stays its own; the per-field provenance tags are the history that travels.
  22. As a future team-plan customer, I want account-scoped portability to exist as a primitive, so that instance consolidation and account moves are possible when multi-user teams arrive.

Implementation Decisions

  • Portability is not backup. Backup stays admin-gated, whole-server, and includes accounts/credentials. Portability is owner-gated, single-account, and excludes users, password hashes, and API keys entirely. Two features, one shared serialization ancestry.
  • A deep portability engine with a two-function interface: export an account to a bundle, import a bundle into an account, returning a report. Everything hard lives behind it: entity serialization, identity re-attribution, provenance preservation, label collision handling, version validation. The API routes, web UI, and CLI are thin callers.
  • The bundle is one zip: a JSON document (bundle format version, source server version, export timestamp, per-entity collections) plus a media directory of content-addressed blobs. Media refs in records resolve against the bundle at import and are written into the target's blob store.
  • Scope of the bundle: tool catalog records, tool instance records, tool table entries, tool sets, machine records, setups (machine–set maps, including ended history rows), binding proposals, labels, and the usage ledger. Record UUIDs are preserved (UUID4 collision risk is negligible and cross-record references — catalog links, bindings, ledger rows — keep working verbatim).
  • Identity remapping: every row's ownership/attribution columns are rewritten to the importing account. Provenance sources inside canonical fields are text and travel untouched — that is the point.
  • Import bypasses the public doors via a privileged, account-scoped write path (the backup-restore precedent): replaying through create/assert/observe would re-stamp provenance and is expressly forbidden. Lane discipline is preserved because the bundle only ever contains data the account already owned.
  • Import requires an empty account (the seed-demo posture: 409 with a pointer to account reset). Merge semantics are a later feature.
  • Label collisions (a code already exists on the target, owned by anyone): the import does not guess — colliding labels are imported blank-less (skipped) and enumerated in the report; everything else proceeds. Printed-URL reality (stickers point at the old host) is stated in the report and the UI copy.
  • Audit: the bundle carries no audit rows. The target server writes its own audit entries recording the import act and its counts.
  • API contract: an owner-gated export endpoint streaming the zip, and an owner-gated import endpoint accepting a multipart upload, returning the report. Bundle format carries its own version, checked on import with honest incompatibility errors.
  • Surfaces: account-page card (download button; upload control with the empty-account requirement explained) and CLI verbs in the client repo (tracked there as a follow-up).
  • Vocabulary: "export" / "import" already exist in the glossary as the no-lock-in escape hatch; new user-facing terms (e.g. "bundle") get glossary entries per the language gate.

Testing Decisions

Tests assert external behavior over HTTP, never engine internals — the proof is what a second server (or account) can see after a round trip, not how the serializer works.

  • Round-trip contract tests (the load-bearing proof): seed a rich account (demo data plus labels, usage observations, media), export, reset, import, then deep-compare every record via the public API — canonical values and provenance sources identical, derived usage totals decompose identically, media bytes byte-identical, label codes resolve.
  • Cross-account import test (the cross-instance proxy): export from account A, import into empty account B on the same server; B sees the full dataset re-attributed to itself; A and B remain isolated; nothing in B's view names A.
  • Engine unit tests: identity remapping coverage across every entity family, label-collision reporting, refusal of malformed/version-incompatible bundles, credential-material exclusion (a bundle can never contain a password hash or key hash — a canary, like the public-page leak test).
  • Refusal tests first (the house post-mortem lesson): non-empty account 409, anonymous 401, cross-account bundle theft impossible (import only ever writes to the caller).
  • Prior art: the existing round-trip loop integration test, the cross-account isolation suite, and the backup authorization tests.
  • New security assumptions (import path privileges, credential exclusion) land as numbered rows with their proving tests in the same commit, per the security-assumptions discipline.

Out of Scope

  • Merge imports into a non-empty account (conflict semantics deserve their own design).
  • Selective export (single tool set / single machine) — this is whole-account portability.
  • Ownership transfer between users (the swap-meet story) — different feature with different trust questions.
  • Automatic label-URL redirection from the old instance to the new one.
  • Audit-log transfer — deliberately excluded, not deferred.
  • CLI implementation — tracked in the client repo once the endpoints exist.
  • Changes to server backup beyond sharing serialization ancestry.

Further Notes

  • The prod deployment currently persists only the database volume; the media blob directory must be bind-mounted before media-bearing bundles matter in production (pre-existing deployment gap, noted 2026-08-04).
  • Gentle-slope check: a casual user never meets this feature until the day they need it, at which point it is one button; the bundle's completeness (ledger, labels, provenance) is the high ceiling. Building it also forces the "what is an account's data, exactly?" enumeration that team features will need later.
  • Design conversation: 2026-08-05 session (labels/usage-ledger release follow-up).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions