-
Notifications
You must be signed in to change notification settings - Fork 3
feat: stack usability and docs updates #306
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 14 commits
8aecf69
ce7d82d
36dc708
a645115
c6dc7d5
2d4c7f3
9185c3d
10ba882
53a8ff0
f8f1426
e161c12
0e8c566
3b3bec3
08b962c
69f88a4
45e40dc
def9f4b
2323e9c
1e6f4dc
46314ba
231cf7f
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,12 @@ | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| --- | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| "@cipherstash/stack": minor | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| --- | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ### Documentation | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| - **TypeDoc**: Improved JSDoc for `Encryption()`, `EncryptOptions`, schema builders (`encryptedTable`, `encryptedColumn`, `encryptedField`, `EncryptedField`, `EncryptedTableColumn`), and `encrypt` / `bulkEncrypt` with clearer `@param`, `@returns`, `@throws`, `@example`, and `@see` links. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| - **README**: Refreshed main repo README and Stack package readme; basic example README now uses `npm install @cipherstash/stack`, CipherStash account and dashboard credentials, and drops Stash CLI references. Added docs badge linking to cipherstash.com/docs. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ### Features | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| - **Logging**: Logger is now used consistently across Stack client interfaces for initialization and operations. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| - **Logging**: Logger is now used consistently across Stack client interfaces for initialization and operations. | |
| - **Logging**: Logger is now used consistently across Stack client interfaces for initialization and operations. Logging configuration is now controlled exclusively via the `STASH_STACK_LOG` environment variable. | |
| ### Breaking changes | |
| - **Removal of `EncryptionClientConfig.logging`**: The `logging` configuration option has been removed from `EncryptionClientConfig`. Logging can no longer be configured programmatically via the `logging` parameter (including `enabled`, `pretty`, and `drain` options) when initializing the Encryption client. Instead, logging must be configured using the `STASH_STACK_LOG` environment variable. | |
| **Migration** | |
| - Before (programmatic logging configuration): | |
| ```ts | |
| import { Encryption } from "@cipherstash/stack"; | |
| const client = new Encryption({ | |
| // other options... | |
| logging: { | |
| enabled: true, | |
| pretty: true, | |
| // drain: someWritableStream, | |
| }, | |
| }); | |
| ``` | |
| - After (environment-based logging configuration): | |
| Set the desired logging mode via `STASH_STACK_LOG` (for example, to enable pretty‑printed logs): | |
| ```bash | |
| export STASH_STACK_LOG=pretty | |
| ``` | |
| Then initialize the Encryption client without the `logging` option: | |
| ```ts | |
| import { Encryption } from "@cipherstash/stack"; | |
| const client = new Encryption({ | |
| // other options... | |
| // logging is now configured via STASH_STACK_LOG | |
| }); | |
| ``` |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -3,7 +3,7 @@ This is the Protect.js repository - End-to-end, per-value encryption for JavaScr | |
| ## Prerequisites | ||
|
|
||
| - **Node.js**: >= 22 (enforced in `package.json` engines) | ||
| - **pnpm**: 9.x (this repo uses pnpm workspaces and catalogs) | ||
| - **pnpm**: 10.14.0 (this repo uses pnpm workspaces and catalogs) | ||
| - Internet access to install the prebuilt native module `@cipherstash/protect-ffi` | ||
|
|
||
| If running integration tests or examples, you will also need CipherStash credentials (see Environment variables below). | ||
|
|
@@ -24,7 +24,7 @@ pnpm run build | |
| pnpm run build:js | ||
| ``` | ||
|
|
||
| Under the hood this uses Turborepo to build `./packages/*` with each package’s `tsup` configuration. | ||
| Under the hood this uses Turborepo to build `./packages/*` with each package's `tsup` configuration. | ||
|
|
||
| ### Dev/watch | ||
|
|
||
|
|
@@ -43,11 +43,11 @@ pnpm test | |
| - Filter to a single package (recommended for fast iteration): | ||
|
|
||
| ```bash | ||
| pnpm --filter @cipherstash/protect test | ||
| pnpm --filter @cipherstash/stack test | ||
| pnpm --filter @cipherstash/nextjs test | ||
| ``` | ||
|
|
||
| Tests use **Vitest**. Many tests talk to the real CipherStash service; they require environment variables. Some tests (e.g., lock context) are skipped if optional tokens aren’t present. | ||
| Tests use **Vitest**. Many tests talk to the real CipherStash service; they require environment variables. Some tests (e.g., lock context) are skipped if optional tokens aren't present. | ||
|
|
||
| ### Environment variables required for runtime/tests | ||
|
|
||
|
|
@@ -63,49 +63,62 @@ CS_CLIENT_ACCESS_KEY= | |
| USER_JWT= | ||
| USER_2_JWT= | ||
|
|
||
| # Optional – CTS endpoint for lock contexts | ||
| CS_CTS_ENDPOINT=https://ap-southeast-2.aws.auth.viturhosted.net | ||
|
|
||
|
||
| # Logging (plaintext is never logged by design) | ||
| PROTECT_LOG_LEVEL=debug|info|error | ||
| STASH_STACK_LOG=debug|info|error # default: error (errors only) | ||
| ``` | ||
|
|
||
| If these variables are missing, tests that require live encryption will fail or be skipped; prefer filtering to specific packages and tests while developing. | ||
|
|
||
| ## Repository Layout | ||
|
|
||
| - `packages/protect`: Core library | ||
| - `src/index.ts`: Public API (`protect`, exports) | ||
| - `src/ffi/index.ts`: `ProtectClient` implementation, bridges to `@cipherstash/protect-ffi` | ||
| - `src/ffi/operations/*`: Encrypt/decrypt/model/bulk/search-terms operations (thenable pattern with optional `.withLockContext()`) | ||
| - `packages/stack`: Main package (`@cipherstash/stack`) containing the encryption client and all integrations | ||
| - Subpath exports: `@cipherstash/stack`, `@cipherstash/stack/schema`, `@cipherstash/stack/identity`, `@cipherstash/stack/secrets`, `@cipherstash/stack/drizzle`, `@cipherstash/stack/supabase`, `@cipherstash/stack/dynamodb`, `@cipherstash/stack/client`, `@cipherstash/stack/types` | ||
| - `packages/protect`: Core encryption library (internal, re-exported via `@cipherstash/stack`) | ||
| - `src/index.ts`: Public API (`Encryption`, exports) | ||
| - `src/ffi/index.ts`: `EncryptionClient` implementation, bridges to `@cipherstash/protect-ffi` | ||
| - `src/ffi/operations/*`: Encrypt/decrypt/model/bulk/query operations (thenable pattern with optional `.withLockContext()`) | ||
| - `__tests__/*`: End-to-end and API contract tests (Vitest) | ||
| - `packages/schema`: Schema builder utilities and types (`csTable`, `csColumn`, `buildEncryptConfig`) | ||
| - `packages/schema`: Schema builder utilities and types (`encryptedTable`, `encryptedColumn`, `encryptedField`) | ||
| - `packages/drizzle`: Drizzle ORM integration (`encryptedType`, `extractEncryptionSchema`, `createEncryptionOperators`) | ||
| - `packages/nextjs`: Next.js helpers and Clerk integration (`./clerk` export) | ||
| - `packages/protect-dynamodb`: DynamoDB helpers for Protect.js | ||
| - `packages/protect-dynamodb`: DynamoDB helpers (`encryptedDynamoDB`) | ||
| - `packages/utils`: Shared config (`utils/config`) and logger (`utils/logger`) | ||
| - `examples/*`: Working apps (basic, drizzle, nextjs-clerk, next-drizzle-mysql, dynamo, hono-supabase) | ||
| - `docs/*`: Concepts, how-to guides (Next.js bundling, SST, npm lockfile v3), reference | ||
| - `skills/*`: Agent skills (`stash-encryption`, `stash-drizzle`, `stash-dynamodb`, `stash-secrets`, `stash-supabase`) | ||
|
|
||
| ## Key Concepts and APIs | ||
|
|
||
| - **Initialization**: `protect({ schemas })` returns an initialized `ProtectClient`. Provide at least one `csTable`. | ||
| - **Schema**: Define tables/columns with `csTable` and `csColumn`. Add `.freeTextSearch().equality().orderAndRange()` to enable searchable encryption on PostgreSQL. | ||
| - **Operations** (all return Result-like objects and support chaining `.withLockContext(lockContext)` when applicable): | ||
| - **Initialization**: `Encryption({ schemas })` returns an initialized `EncryptionClient`. Provide at least one `encryptedTable`. | ||
| - **Schema**: Define tables/columns with `encryptedTable` and `encryptedColumn` from `@cipherstash/stack/schema`. Add `.freeTextSearch().equality().orderAndRange()` to enable searchable encryption on PostgreSQL. Use `.searchableJson()` for encrypted JSONB queries. Use `encryptedField` for nested object encryption (DynamoDB). | ||
| - **Operations** (all return Result-like objects and support chaining `.withLockContext(lockContext)` and `.audit()` when applicable): | ||
| - `encrypt(plaintext, { table, column })` | ||
| - `decrypt(encryptedPayload)` | ||
| - `encryptModel(model, table)` / `decryptModel(model)` | ||
| - `bulkEncrypt(plaintexts[], { table, column })` / `bulkDecrypt(encrypted[])` | ||
| - `bulkEncryptModels(models[], table)` / `bulkDecryptModels(models[])` | ||
| - `createSearchTerms(terms)` for searchable queries | ||
| - **Identity-aware encryption**: Use `LockContext` from `@cipherstash/protect/identify` and chain `.withLockContext()` on operations. Same context must be used for both encrypt and decrypt. | ||
| - `encryptQuery(value, { table, column, queryType?, returnType? })` for searchable queries | ||
| - `encryptQuery(terms[])` for batch query encryption | ||
| - **Identity-aware encryption**: Use `LockContext` from `@cipherstash/stack/identity` and chain `.withLockContext()` on operations. Same context must be used for both encrypt and decrypt. | ||
| - **Integrations**: | ||
| - **Drizzle ORM**: `encryptedType`, `extractEncryptionSchema`, `createEncryptionOperators` from `@cipherstash/stack/drizzle` | ||
| - **Supabase**: `encryptedSupabase` from `@cipherstash/stack/supabase` | ||
| - **DynamoDB**: `encryptedDynamoDB` from `@cipherstash/stack/dynamodb` | ||
| - **Secrets management**: `Secrets` class from `@cipherstash/stack/secrets` for encrypted secret storage and retrieval. | ||
|
|
||
| ## Critical Gotchas (read before coding) | ||
|
|
||
| - **Native Node.js module**: Protect.js relies on `@cipherstash/protect-ffi` (Node-API). It must be loaded via native Node.js `require`. Do NOT bundle this module; configure bundlers to externalize it. | ||
| - **Native Node.js module**: `@cipherstash/stack` relies on `@cipherstash/protect-ffi` (Node-API). It must be loaded via native Node.js `require`. Do NOT bundle this module; configure bundlers to externalize it. | ||
| - Next.js: see `docs/how-to/nextjs-external-packages.md` | ||
| - SST/Serverless: see `docs/how-to/sst-external-packages.md` | ||
| - npm lockfile v3 on Linux: see `docs/how-to/npm-lockfile-v3.md` | ||
| - **Do not log plaintext**: The library never logs plaintext by design. Don’t add logs that risk leaking sensitive data. | ||
| - **Result shape is contract**: Operations return `{ data }` or `{ failure }`. Preserve this shape and error `type` values in `ProtectErrorTypes`. | ||
| - **Encrypted payload shape is contract**: Keys like `c` in the EQL payload are validated by tests and downstream tools. Don’t change them. | ||
| - **Exports must support ESM and CJS**: Each package’s `exports` maps must keep both `import` and `require` fields. Don’t remove CJS. | ||
| - **Do not log plaintext**: The library never logs plaintext by design. Don't add logs that risk leaking sensitive data. | ||
| - **Result shape is contract**: Operations return `{ data }` or `{ failure }`. Preserve this shape and error `type` values in `EncryptionErrorTypes`. | ||
| - **Encrypted payload shape is contract**: Keys like `c` in the EQL payload are validated by tests and downstream tools. Don't change them. | ||
| - **Exports must support ESM and CJS**: Each package's `exports` maps must keep both `import` and `require` fields. Don't remove CJS. | ||
|
|
||
| ## Development Workflow | ||
|
|
||
|
|
@@ -127,25 +140,25 @@ pnpm changeset:publish | |
|
|
||
| ### Writing tests | ||
|
|
||
| - Use Vitest with `.test.ts` files under each package’s `__tests__/`. | ||
| - Use Vitest with `.test.ts` files under each package's `__tests__/`. | ||
| - Import `dotenv/config` at the top when tests need environment variables. | ||
| - Prefer testing via the public API. Avoid reaching into private internals. | ||
| - Some tests have larger timeouts (e.g., 30s) to accommodate network calls. | ||
|
|
||
| ## Bundling and Deployment Notes | ||
|
|
||
| - When integrating into frameworks/build tools, ensure native modules are externalized and loaded via Node’s runtime require. | ||
| - When integrating into frameworks/build tools, ensure native modules are externalized and loaded via Node's runtime require. | ||
| - For Next.js, configure `serverExternalPackages` as documented in `docs/how-to/nextjs-external-packages.md`. | ||
| - For serverless/Linux targets with npm lockfile v3, see `docs/how-to/npm-lockfile-v3.md` to avoid runtime load errors. | ||
|
|
||
| ## Adding Features Safely (LLM checklist) | ||
|
|
||
| 1. Identify the target package(s) in `packages/*` and confirm whether changes affect public APIs or payload shapes. | ||
| 2. If modifying `packages/protect` operations or `ProtectClient`, ensure: | ||
| 2. If modifying `packages/protect` operations or `EncryptionClient`, ensure: | ||
| - The Result contract and error type strings remain stable. | ||
| - `.withLockContext()` remains available for affected operations. | ||
| - ESM/CJS exports continue to work (don’t break `require`). | ||
| 3. If changing schema behavior (`packages/schema`), update type definitions and ensure `buildEncryptConfig` still validates with Zod in `ProtectClient.init`. | ||
| - ESM/CJS exports continue to work (don't break `require`). | ||
| 3. If changing schema behavior (`packages/schema`), update type definitions and ensure validation still works in `EncryptionClient.init`. | ||
| 4. Add/extend tests in the same package. For features that require live credentials, guard with env checks or provide mock-friendly paths. | ||
| 5. Run: | ||
| - `pnpm run code:fix` | ||
|
|
@@ -157,17 +170,22 @@ pnpm changeset:publish | |
|
|
||
| - `README.md` for quickstart and feature overview | ||
| - `docs/concepts/searchable-encryption.md` | ||
| - `docs/concepts/aws-kms-vs-cipherstash-comparison.md` | ||
| - `docs/reference/schema.md` | ||
| - `docs/reference/searchable-encryption-postgres.md` | ||
| - `docs/reference/configuration.md` | ||
| - `docs/reference/identity.md` | ||
| - `docs/reference/secrets.md` | ||
| - `docs/reference/dynamodb.md` | ||
| - `docs/reference/supabase-sdk.md` | ||
| - `docs/reference/drizzle/drizzle.md` | ||
| - `docs/how-to/nextjs-external-packages.md` | ||
| - `docs/how-to/sst-external-packages.md` | ||
| - `docs/how-to/npm-lockfile-v3.md` | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| - Module load errors on Linux/serverless: review the npm lockfile v3 guide. | ||
| - Can’t decrypt after encrypting with a lock context: ensure the exact same lock context is provided to decrypt. | ||
| - Can't decrypt after encrypting with a lock context: ensure the exact same lock context is provided to decrypt. | ||
| - Tests failing due to missing credentials: provide `CS_*` env vars; lock-context tests are skipped without `USER_JWT`. | ||
| - Performance testing: prefer bulk operations (`bulkEncrypt*` / `bulkDecrypt*`) to exercise ZeroKMS bulk speed. | ||
|
|
||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The Secrets client configuration has a breaking change:
apiKeywas renamed toaccessKey. While this improves consistency with the rest of the codebase (CS_CLIENT_ACCESS_KEY), this breaking change should be documented in the changeset file. Consider adding a note about this breaking change and, if this warrants a major version bump, update the changeset type accordingly.