Skip to content

Commit 616d13a

Browse files
authored
fix: redact streamed tool arguments safely (#37)
* test: add tool-call streaming fixtures * fix: redact streamed tool arguments * fix: preserve streamed tool JSON Caller-audit: Sources/PastewatchCore/SSEStreamRelay.swift:381 -- VERIFIED typed failures terminate before further socket writes Sources/PastewatchCore/CurlHTTPClient.swift:845 -- VERIFIED Linux relay preserves the same fail-closed ordering * fix: scan escaped truncated tool JSON Caller-audit: Sources/PastewatchCore/ToolCallStreamRedactor.swift:235 -- unaffected; nil remains the typed tool-byte-mapping termination path * chore: bump version to 0.33.0
1 parent b56fb8f commit 616d13a

16 files changed

Lines changed: 2313 additions & 148 deletions

‎CHANGELOG.md‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,32 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
## [Unreleased]
99

10+
## [0.33.0] - 2026-07-19
11+
12+
### Added
13+
14+
- The API proxy now redacts secrets inside **streamed tool-call arguments**. A shared
15+
event-aware relay reassembles Anthropic `input_json_delta.partial_json` and
16+
OpenAI-compatible/LiteLLM `tool_calls[].function.arguments` fragments across SSE frames
17+
before scanning, so a secret split across frame boundaries — or carried in a JSON object
18+
key, or spelled with `\u` escapes — is still caught. Redaction preserves the exact JSON
19+
bytes outside authorized replacements; a fragment that cannot be redacted while keeping
20+
valid JSON is blocked fail-closed rather than forwarded.
21+
- `pastewatch-cli proxy --debug-stream-dump <path>` captures raw upstream frames,
22+
transformed output, and mutation decisions as owner-only (`0600`) JSONL for local
23+
protocol diagnosis. Opt-in only, requires the default `per_sse_event` mode (startup
24+
fails otherwise), contains unredacted secrets by design, and prints a warning even under
25+
`--quiet`. The dump file is opened no-follow to prevent symlink path substitution.
26+
27+
### Fixed
28+
29+
- A `\u`-escaped secret carried in a **truncated or malformed** tool-call JSON payload is
30+
no longer forwarded unredacted: complete escaped tokens are decoded and scanned even
31+
when the aggregate JSON never parses, and any escape that cannot be mapped and scanned
32+
blocks the frame fail-closed.
33+
- `ProxyServer` serializes the shared `ISO8601DateFormatter`'s lazy cache initialization on
34+
Linux (swift-corelibs-foundation) so concurrent audit-log timestamping is thread-safe.
35+
1036
## [0.32.0] - 2026-07-17
1137

1238
### Added

‎README.md‎

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Pastewatch
22
[![Stable](https://img.shields.io/badge/status-stable-brightgreen)](https://github.com/ppiankov/pastewatch/releases)
3-
[![Version](https://img.shields.io/badge/version-0.32.0-blue)](https://github.com/ppiankov/pastewatch/releases/tag/v0.32.0)
3+
[![Version](https://img.shields.io/badge/version-0.33.0-blue)](https://github.com/ppiankov/pastewatch/releases/tag/v0.33.0)
44
[![License: MIT](https://img.shields.io/badge/license-MIT-yellow)](LICENSE)
55
[![CI](https://github.com/ppiankov/pastewatch/actions/workflows/ci.yml/badge.svg)](https://github.com/ppiankov/pastewatch/actions/workflows/ci.yml)
66
[![ANCC](https://img.shields.io/badge/ANCC-compliant-brightgreen)](https://ancc.dev)
@@ -479,7 +479,11 @@ pastewatch-cli launch --audit-log /tmp/pw-audit.log -- claude
479479
[2026-03-16T11:36:56Z] PROXY REDACTED 3 secret(s) in /v1/messages
480480
```
481481

482-
**Streaming response mode.** `responseStreamingRedactionMode=buffer` is a compatibility mode for full-response buffering. It does not scan buffered response bodies for secrets; use the default `per_sse_event` mode when response redaction is required.
482+
**Streaming response mode.** `responseStreamingRedactionMode=buffer` is a compatibility mode for full-response buffering. It does not scan buffered response bodies for secrets; use the default `per_sse_event` mode when response redaction is required. The event-aware relay reassembles Anthropic `input_json_delta.partial_json` and OpenAI-compatible/LiteLLM `tool_calls[].function.arguments` fragments before scanning, then preserves all frame bytes outside authorized replacements. This response support does not change request admission: OpenAI-shaped request bodies are still refused.
483+
484+
Response streaming has no authoritative catalog of exact local secret values. It mutates intrinsically identifiable formats and operator-approved custom rules; adding exact-value response matching requires a separately designed local secret source and lifecycle.
485+
486+
For local protocol diagnosis only, `pastewatch-cli proxy --debug-stream-dump <path>` writes raw input frames, transformed output, and mutation decisions as owner-only JSONL. It requires the default `per_sse_event` mode so each record reflects the actual frame decision; startup fails instead of producing an incomplete dump in `raw_stream` or `buffer` mode. The file contains unredacted secrets by design, is disabled unless the option is supplied, and prints a warning even with `--quiet`. Delete it securely after diagnosis and never attach it to an issue or commit.
483487

484488
### MCP Server - Redacted Read/Write
485489

@@ -784,7 +788,7 @@ Works with any comment style (`#`, `//`, `/* */`).
784788
# .pre-commit-config.yaml
785789
repos:
786790
- repo: https://github.com/ppiankov/pastewatch
787-
rev: v0.32.0
791+
rev: v0.33.0
788792
hooks:
789793
- id: pastewatch
790794
```
@@ -997,7 +1001,7 @@ Do not pretend it guarantees compliance or safety.
9971001

9981002
## Project Status
9991003

1000-
**Status: Stable, feature-complete** · **v0.32.0** · Accepting compatibility and bug fixes only
1004+
**Status: Stable, feature-complete** · **v0.33.0** · Accepting compatibility and bug fixes only
10011005

10021006
| Milestone | Status |
10031007
|-----------|--------|

‎Sources/PastewatchCLI/ProxyCommand.swift‎

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,12 @@ func proxyShutdownExitCode(didStart: Bool) -> Int32 {
3838
didStart ? 0 : proxyInterruptedExitCode
3939
}
4040

41+
// WO-514: raw stream capture must remain conspicuous even when normal logs are quiet.
42+
func streamDebugDumpWarning(path: String?) -> String? {
43+
guard let path else { return nil }
44+
return "WARNING: --debug-stream-dump writes raw streaming data with secrets to local file: \(path)\n"
45+
}
46+
4147
struct Proxy: ParsableCommand {
4248
static let configuration = CommandConfiguration(
4349
abstract: "Start API proxy that scans and redacts secrets from outbound requests"
@@ -58,6 +64,10 @@ struct Proxy: ParsableCommand {
5864
@Option(name: .long, help: "Audit log file path")
5965
var auditLog: String?
6066

67+
// WO-514: raw capture requires an explicit CLI path and has no config default.
68+
@Option(name: .long, help: "Write raw streaming frames and redaction decisions to a local file")
69+
var debugStreamDump: String?
70+
6171
@Flag(name: .long, inversion: .prefixedNo, help: "Inject alert into response when secrets are redacted")
6272
var alert: Bool = true
6373

@@ -92,6 +102,7 @@ struct Proxy: ParsableCommand {
92102

93103
let config = PastewatchConfig.resolve()
94104
let compiledCustomRules = try requireValidProxyCustomRules(config)
105+
let streamDebugSink = try debugStreamDump.map { try StreamDebugSink(path: $0) }
95106
let server = ProxyServer(
96107
port: port,
97108
upstream: upstreamURL,
@@ -103,7 +114,8 @@ struct Proxy: ParsableCommand {
103114
injectAlert: alert,
104115
quietLog: quiet,
105116
caCertPath: caCert,
106-
insecureTLS: insecure
117+
insecureTLS: insecure,
118+
streamDebugSink: streamDebugSink
107119
)
108120

109121
FileHandle.standardError.write(Data("pastewatch proxy listening on http://127.0.0.1:\(port)\n".utf8))
@@ -113,6 +125,9 @@ struct Proxy: ParsableCommand {
113125
}
114126
FileHandle.standardError.write(Data("severity: \(severity.rawValue)\n".utf8))
115127
FileHandle.standardError.write(Data("alert-injection: \(alert ? "on" : "off")\n".utf8))
128+
if let warning = streamDebugDumpWarning(path: debugStreamDump) {
129+
FileHandle.standardError.write(Data(warning.utf8))
130+
}
116131
if let warning = ProxyServer.bufferModeWarning(config: config, quiet: quiet) {
117132
FileHandle.standardError.write(Data(warning.utf8))
118133
}

0 commit comments

Comments
 (0)