Skip to content
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/lazy-rocks-look.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@cipherstash/stack": minor
---

Improve TypeDoc interfaces.
76 changes: 47 additions & 29 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ This is the Protect.js repository - End-to-end, per-value encryption for JavaScr
## Prerequisites

- **Node.js**: >= 22 (enforced in `package.json` engines)
- **pnpm**: 9.x (this repo uses pnpm workspaces and catalogs)
- **pnpm**: 10.14.0 (this repo uses pnpm workspaces and catalogs)
- Internet access to install the prebuilt native module `@cipherstash/protect-ffi`

If running integration tests or examples, you will also need CipherStash credentials (see Environment variables below).
Expand All @@ -24,7 +24,7 @@ pnpm run build
pnpm run build:js
```

Under the hood this uses Turborepo to build `./packages/*` with each packages `tsup` configuration.
Under the hood this uses Turborepo to build `./packages/*` with each package's `tsup` configuration.

### Dev/watch

Expand All @@ -43,11 +43,11 @@ pnpm test
- Filter to a single package (recommended for fast iteration):

```bash
pnpm --filter @cipherstash/protect test
pnpm --filter @cipherstash/stack test
pnpm --filter @cipherstash/nextjs test
```

Tests use **Vitest**. Many tests talk to the real CipherStash service; they require environment variables. Some tests (e.g., lock context) are skipped if optional tokens arent present.
Tests use **Vitest**. Many tests talk to the real CipherStash service; they require environment variables. Some tests (e.g., lock context) are skipped if optional tokens aren't present.

### Environment variables required for runtime/tests

Expand All @@ -63,49 +63,62 @@ CS_CLIENT_ACCESS_KEY=
USER_JWT=
USER_2_JWT=

# Optional – CTS endpoint for lock contexts
CS_CTS_ENDPOINT=https://ap-southeast-2.aws.auth.viturhosted.net

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this needed? CS_CTS_ENDPOINT should never be set outside of testing.

# Logging (plaintext is never logged by design)
PROTECT_LOG_LEVEL=debug|info|error
STASH_LOG_LEVEL=debug|info|warn|error
```

If these variables are missing, tests that require live encryption will fail or be skipped; prefer filtering to specific packages and tests while developing.

## Repository Layout

- `packages/protect`: Core library
- `src/index.ts`: Public API (`protect`, exports)
- `src/ffi/index.ts`: `ProtectClient` implementation, bridges to `@cipherstash/protect-ffi`
- `src/ffi/operations/*`: Encrypt/decrypt/model/bulk/search-terms operations (thenable pattern with optional `.withLockContext()`)
- `packages/stack`: Main package (`@cipherstash/stack`) containing the encryption client and all integrations
- Subpath exports: `@cipherstash/stack`, `@cipherstash/stack/schema`, `@cipherstash/stack/identity`, `@cipherstash/stack/secrets`, `@cipherstash/stack/drizzle`, `@cipherstash/stack/supabase`, `@cipherstash/stack/dynamodb`, `@cipherstash/stack/client`, `@cipherstash/stack/types`
- `packages/protect`: Core encryption library (internal, re-exported via `@cipherstash/stack`)
- `src/index.ts`: Public API (`Encryption`, exports)
- `src/ffi/index.ts`: `EncryptionClient` implementation, bridges to `@cipherstash/protect-ffi`
- `src/ffi/operations/*`: Encrypt/decrypt/model/bulk/query operations (thenable pattern with optional `.withLockContext()`)
- `__tests__/*`: End-to-end and API contract tests (Vitest)
- `packages/schema`: Schema builder utilities and types (`csTable`, `csColumn`, `buildEncryptConfig`)
- `packages/schema`: Schema builder utilities and types (`encryptedTable`, `encryptedColumn`, `encryptedField`)
- `packages/drizzle`: Drizzle ORM integration (`encryptedType`, `extractEncryptionSchema`, `createEncryptionOperators`)
- `packages/nextjs`: Next.js helpers and Clerk integration (`./clerk` export)
- `packages/protect-dynamodb`: DynamoDB helpers for Protect.js
- `packages/protect-dynamodb`: DynamoDB helpers (`encryptedDynamoDB`)
- `packages/utils`: Shared config (`utils/config`) and logger (`utils/logger`)
- `examples/*`: Working apps (basic, drizzle, nextjs-clerk, next-drizzle-mysql, dynamo, hono-supabase)
- `docs/*`: Concepts, how-to guides (Next.js bundling, SST, npm lockfile v3), reference
- `skills/*`: Agent skills (`stash-encryption`, `stash-drizzle`, `stash-dynamodb`, `stash-secrets`, `stash-supabase`)

## Key Concepts and APIs

- **Initialization**: `protect({ schemas })` returns an initialized `ProtectClient`. Provide at least one `csTable`.
- **Schema**: Define tables/columns with `csTable` and `csColumn`. Add `.freeTextSearch().equality().orderAndRange()` to enable searchable encryption on PostgreSQL.
- **Operations** (all return Result-like objects and support chaining `.withLockContext(lockContext)` when applicable):
- **Initialization**: `Encryption({ schemas })` returns an initialized `EncryptionClient`. Provide at least one `encryptedTable`.
- **Schema**: Define tables/columns with `encryptedTable` and `encryptedColumn` from `@cipherstash/stack/schema`. Add `.freeTextSearch().equality().orderAndRange()` to enable searchable encryption on PostgreSQL. Use `.searchableJson()` for encrypted JSONB queries. Use `encryptedField` for nested object encryption (DynamoDB).
- **Operations** (all return Result-like objects and support chaining `.withLockContext(lockContext)` and `.audit()` when applicable):
- `encrypt(plaintext, { table, column })`
- `decrypt(encryptedPayload)`
- `encryptModel(model, table)` / `decryptModel(model)`
- `bulkEncrypt(plaintexts[], { table, column })` / `bulkDecrypt(encrypted[])`
- `bulkEncryptModels(models[], table)` / `bulkDecryptModels(models[])`
- `createSearchTerms(terms)` for searchable queries
- **Identity-aware encryption**: Use `LockContext` from `@cipherstash/protect/identify` and chain `.withLockContext()` on operations. Same context must be used for both encrypt and decrypt.
- `encryptQuery(value, { table, column, queryType?, returnType? })` for searchable queries
- `encryptQuery(terms[])` for batch query encryption
- **Identity-aware encryption**: Use `LockContext` from `@cipherstash/stack/identity` and chain `.withLockContext()` on operations. Same context must be used for both encrypt and decrypt.
- **Integrations**:
- **Drizzle ORM**: `encryptedType`, `extractEncryptionSchema`, `createEncryptionOperators` from `@cipherstash/stack/drizzle`
- **Supabase**: `encryptedSupabase` from `@cipherstash/stack/supabase`
- **DynamoDB**: `encryptedDynamoDB` from `@cipherstash/stack/dynamodb`
- **Secrets management**: `Secrets` class from `@cipherstash/stack/secrets` for encrypted secret storage and retrieval.

## Critical Gotchas (read before coding)

- **Native Node.js module**: Protect.js relies on `@cipherstash/protect-ffi` (Node-API). It must be loaded via native Node.js `require`. Do NOT bundle this module; configure bundlers to externalize it.
- **Native Node.js module**: `@cipherstash/stack` relies on `@cipherstash/protect-ffi` (Node-API). It must be loaded via native Node.js `require`. Do NOT bundle this module; configure bundlers to externalize it.
- Next.js: see `docs/how-to/nextjs-external-packages.md`
- SST/Serverless: see `docs/how-to/sst-external-packages.md`
- npm lockfile v3 on Linux: see `docs/how-to/npm-lockfile-v3.md`
- **Do not log plaintext**: The library never logs plaintext by design. Dont add logs that risk leaking sensitive data.
- **Result shape is contract**: Operations return `{ data }` or `{ failure }`. Preserve this shape and error `type` values in `ProtectErrorTypes`.
- **Encrypted payload shape is contract**: Keys like `c` in the EQL payload are validated by tests and downstream tools. Dont change them.
- **Exports must support ESM and CJS**: Each packages `exports` maps must keep both `import` and `require` fields. Dont remove CJS.
- **Do not log plaintext**: The library never logs plaintext by design. Don't add logs that risk leaking sensitive data.
- **Result shape is contract**: Operations return `{ data }` or `{ failure }`. Preserve this shape and error `type` values in `EncryptionErrorTypes`.
- **Encrypted payload shape is contract**: Keys like `c` in the EQL payload are validated by tests and downstream tools. Don't change them.
- **Exports must support ESM and CJS**: Each package's `exports` maps must keep both `import` and `require` fields. Don't remove CJS.

## Development Workflow

Expand All @@ -127,25 +140,25 @@ pnpm changeset:publish

### Writing tests

- Use Vitest with `.test.ts` files under each packages `__tests__/`.
- Use Vitest with `.test.ts` files under each package's `__tests__/`.
- Import `dotenv/config` at the top when tests need environment variables.
- Prefer testing via the public API. Avoid reaching into private internals.
- Some tests have larger timeouts (e.g., 30s) to accommodate network calls.

## Bundling and Deployment Notes

- When integrating into frameworks/build tools, ensure native modules are externalized and loaded via Nodes runtime require.
- When integrating into frameworks/build tools, ensure native modules are externalized and loaded via Node's runtime require.
- For Next.js, configure `serverExternalPackages` as documented in `docs/how-to/nextjs-external-packages.md`.
- For serverless/Linux targets with npm lockfile v3, see `docs/how-to/npm-lockfile-v3.md` to avoid runtime load errors.

## Adding Features Safely (LLM checklist)

1. Identify the target package(s) in `packages/*` and confirm whether changes affect public APIs or payload shapes.
2. If modifying `packages/protect` operations or `ProtectClient`, ensure:
2. If modifying `packages/protect` operations or `EncryptionClient`, ensure:
- The Result contract and error type strings remain stable.
- `.withLockContext()` remains available for affected operations.
- ESM/CJS exports continue to work (dont break `require`).
3. If changing schema behavior (`packages/schema`), update type definitions and ensure `buildEncryptConfig` still validates with Zod in `ProtectClient.init`.
- ESM/CJS exports continue to work (don't break `require`).
3. If changing schema behavior (`packages/schema`), update type definitions and ensure validation still works in `EncryptionClient.init`.
4. Add/extend tests in the same package. For features that require live credentials, guard with env checks or provide mock-friendly paths.
5. Run:
- `pnpm run code:fix`
Expand All @@ -157,17 +170,22 @@ pnpm changeset:publish

- `README.md` for quickstart and feature overview
- `docs/concepts/searchable-encryption.md`
- `docs/concepts/aws-kms-vs-cipherstash-comparison.md`
- `docs/reference/schema.md`
- `docs/reference/searchable-encryption-postgres.md`
- `docs/reference/configuration.md`
- `docs/reference/identity.md`
- `docs/reference/secrets.md`
- `docs/reference/dynamodb.md`
- `docs/reference/supabase-sdk.md`
- `docs/reference/drizzle/drizzle.md`
- `docs/how-to/nextjs-external-packages.md`
- `docs/how-to/sst-external-packages.md`
- `docs/how-to/npm-lockfile-v3.md`

## Troubleshooting

- Module load errors on Linux/serverless: review the npm lockfile v3 guide.
- Cant decrypt after encrypting with a lock context: ensure the exact same lock context is provided to decrypt.
- Can't decrypt after encrypting with a lock context: ensure the exact same lock context is provided to decrypt.
- Tests failing due to missing credentials: provide `CS_*` env vars; lock-context tests are skipped without `USER_JWT`.
- Performance testing: prefer bulk operations (`bulkEncrypt*` / `bulkDecrypt*`) to exercise ZeroKMS bulk speed.


Loading