fix(hbsig): match HyperBEAM httpsig@1.0 verification format

UPSTREAM_ISSUE_TEMPLATE.md

@@ -0,0 +1,185 @@
+# Upstream Issue/PR for ArweaveOasis/wao
+
+## Issue Title
+```
+hbsig: HTTP signature verification fails with HyperBEAM httpsig@1.0
+```
+
+## Issue Body
+
+Copy everything below to create an issue at **https://github.com/ArweaveOasis/wao/issues/new**
+
+---
+
+## Problem
+
+The `hbsig` library produces HTTP message signatures that fail verification on HyperBEAM with `{process_not_verified, ...}` errors. Both RSA-PSS-SHA512 and HMAC-SHA256 commitments fail.
+
+## Root Causes
+
+After debugging against HyperBEAM source code (`dev_codec_httpsig.erl`), I identified several format mismatches:
+
+### 1. HMAC keyid/key format mismatch
+
+**Current behavior:**
+```javascript
+keyid: "hmac:ao" // or similar
+```
+
+**HyperBEAM expects:**
+```javascript
+keyid: "constant:ao"
+key: "constant:ao"
+```
+
+See `dev_codec_httpsig_keyid.erl` line 24:
+```erlang
+-define(HMAC_DEFAULT_KEY, <<"constant:ao">>).
+```
+
+### 2. Missing `@` prefix for derived components in signature-params
+
+HyperBEAM's `signature_params_line()` function calls `add_derived_specifiers()` which adds `@` prefix to derived components like `authority`, `path`, `method`, etc.
+
+**Current behavior:**
+```
+"@signature-params": ("authority" "data-protocol" ...);alg="...";keyid="..."
+```
+
+**HyperBEAM expects:**
+```
+"@signature-params": ("@authority" "data-protocol" ...);alg="...";keyid="..."
+```
+
+The component LINES should NOT have `@` prefix, but the params line MUST have it for derived components (authority, path, method, target-uri, scheme, request-target, query, query-param).
+
+### 3. Committed field should not have `@` prefix
+
+When sending the commitment, the `committed` array should contain `["authority", ...]` not `["@authority", ...]`. HyperBEAM adds the `@` back via `add_derived_specifiers()` during verification.
+
+### 4. commit() path option not passed correctly
+
+The `commit()` function passes the full options object to the signer instead of extracting the `path` option, causing path to always be included in signatures.
+
+## Reproduction
+
+```javascript
+import { createSigner } from '@permaweb/aoconnect';
+import { commit, signer as createSignerFn } from 'wao/hbsig';
+
+const jwk = /* your wallet */;
+const HB_URL = 'http://your-hyperbeam:8734';
+
+const aoSigner = createSigner(jwk, HB_URL);
+const signerFn = createSignerFn({ signer: aoSigner, url: HB_URL });
+
+const processMsg = {
+  "Data-Protocol": "ao",
+  Variant: "ao.TN.1",
+  device: "process@1.0",
+  Type: "Process",
+  Authority: "scheduler-address",
+  Scheduler: "scheduler-address",
+  Module: "module-id",
+  "execution-device": "genesis-wasm@1.0",
+  "random-seed": "seed-123",
+};
+
+const committed = await commit(processMsg, { signer: signerFn });
+// This commitment will fail verification on HyperBEAM with:
+// {process_not_verified, ...}
+```
+
+## Environment
+
+- HyperBEAM version: 1230ubn1 (production)
+- wao version: latest from main branch
+
+## Proposed Fix
+
+I have a working fix: https://github.com/credentum/wao/commit/97af99a
+
+The fix modifies:
+- `hbsig/src/id.js` - HMAC keyid/key format and derived component handling
+- `hbsig/src/send.js` - Add `@` prefix to derived components in params line
+- `hbsig/src/commit.js` - Strip `@` prefix from committed field, fix path option
+- `hbsig/src/httpsig.js` and `structured.js` - Preserve key casing for commitment IDs
+
+**Test Results** (against HyperBEAM production):
+- ✅ RSA-PSS-SHA512 commitments verify successfully
+- ✅ HMAC-SHA256 commitments verify successfully
+- ✅ HMAC ID matches expected value
+- ✅ Process spawn via commit() function succeeds
+
+Happy to submit a PR if this approach looks correct!
+
+---
+
+## How to Create the Upstream PR
+
+After creating the issue, to submit a PR:
+
+### Option 1: GitHub Web UI
+
+1. Go to https://github.com/ArweaveOasis/wao
+2. Click "Pull requests" → "New pull request"
+3. Click "compare across forks"
+4. Set:
+   - Base repository: `ArweaveOasis/wao`
+   - Base: `main`
+   - Head repository: `credentum/wao`
+   - Compare: `main`
+5. Create the PR
+
+### Option 2: GitHub CLI (if you have permissions)
+
+```bash
+gh pr create --repo ArweaveOasis/wao \
+  --head credentum:main \
+  --title "fix(hbsig): match HyperBEAM httpsig@1.0 verification format" \
+  --body-file UPSTREAM_PR_BODY.md
+```
+
+---
+
+## PR Title
+```
+fix(hbsig): match HyperBEAM httpsig@1.0 verification format
+```
+
+## PR Body
+
+```markdown
+## Summary
+
+Fixes HTTP message signature verification to match HyperBEAM's httpsig@1.0 commitment device format. Previously, commitments created by hbsig failed verification with `{process_not_verified, ...}` errors.
+
+Closes #[ISSUE_NUMBER]
+
+## Changes
+
+### 1. HMAC ID computation (`id.js`)
+- Add `@` prefix to derived components in signature-params line
+- Use `"constant:ao"` for both keyid and key
+- Match HyperBEAM's `add_derived_specifiers()` behavior
+
+### 2. RSA signature (`send.js`)
+- Add `@` prefix to derived components in signature-params line (not component lines)
+- Derived components: authority, path, method, target-uri, scheme, request-target, query, query-param
+
+### 3. Commitment format (`commit.js`)
+- Strip `@` prefix from committed field (HyperBEAM adds it back)
+- Fix path option passing (default to `path: false`)
+- Convert signature to base64url
+
+### 4. Commitment ID (`httpsig.js`, `structured.js`)
+- Preserve original key casing for commitment IDs
+
+## Test Results
+
+Tested against HyperBEAM production (version 1230ubn1):
+- ✅ RSA-PSS-SHA512 commitments verify successfully
+- ✅ HMAC-SHA256 commitments verify successfully
+- ✅ HMAC ID matches expected value
+- ✅ Process spawn via `commit()` function succeeds
+```

hbsig/src/commit.js

@@ -2,10 +2,28 @@ import { id, base, hashpath, rsaid, hmacid } from "./id.js"
 import { toAddr } from "./utils.js"
 import { extractPubKey } from "./signer-utils.js"
 import { verify } from "./signer-utils.js"
+import { decodeSigInput } from "./parser.js"
+import base64url from "base64url"
 
-// todo: handle @
+/**
+ * Create a commitment structure for HyperBEAM
+ *
+ * FIX: Updated to match HyperBEAM's expected commitment format:
+ * - Added 'type' field (algorithm name)
+ * - Added 'keyid' field (publickey:base64 for RSA, constant:ao for HMAC)
+ * - Added 'committed' field (list of signed components)
+ * - Signature is raw base64, not RFC 9421 structured field format
+ * - HMAC signature equals HMAC ID (deterministic)
+ * - Removed signature-input from commitment body (only needed in HTTP headers)
+ *
+ * @see https://github.com/ArweaveOasis/wao/issues/XXX
+ */
 export const commit = async (obj, opts) => {
-  const msg = await opts.signer(obj, opts)
+  // CRITICAL: Pass path: false to prevent signing @path component
+  // HyperBEAM commitment verification doesn't expect path in the committed fields
+  // when spawning a process (only for messages to a process)
+  const signingOpts = { path: opts.path !== undefined ? opts.path : false }
+  const msg = await opts.signer(obj, signingOpts)
   const {
     decodedSignatureInput: { components },
   } = await verify(msg)
@@ -16,9 +34,13 @@ export const commit = async (obj, opts) => {
   const inlineBodyKey = msg.headers["inline-body-key"]
 
   // Build body from components
+  // Note: signature-input has @-prefixed derived components (e.g., @authority)
+  // but the actual headers and body keys don't have the @ prefix
   for (const v of components) {
-    const key = v === "@path" ? "path" : v
-    body[key] = msg.headers[key]
+    // Strip @ prefix from derived components to get the actual header key
+    const headerKey = v.startsWith('@') ? v.slice(1) : v
+    // For body, use the key without @ prefix
+    body[headerKey] = msg.headers[headerKey]
   }
 
   // Handle body resolution
@@ -47,16 +69,75 @@ export const commit = async (obj, opts) => {
   const rsaId = rsaid(msg.headers)
   const pub = extractPubKey(msg.headers)
   const committer = toAddr(pub.toString("base64"))
-  const meta = { alg: "rsa-pss-sha512", "commitment-device": "httpsig@1.0" }
-  const meta2 = { alg: "hmac-sha256", "commitment-device": "httpsig@1.0" }
-  const sigs = {
-    signature: msg.headers.signature,
-    "signature-input": msg.headers["signature-input"],
+
+  // Parse signature-input to extract params (keyid, tag, alg)
+  const signatureInputHeader = msg.headers["signature-input"] || msg.headers["Signature-Input"]
+  const decodedSigs = decodeSigInput(signatureInputHeader)
+  const sigEntries = Object.entries(decodedSigs || {})
+  const [sigName, sigData] = sigEntries[0] || [null, {}]
+  const sigParams = sigData?.params || {}
+
+  // Parse RFC 9421 structured field signature to extract raw base64 value
+  // RFC 9421 format: "label=:base64value:" - HyperBEAM expects just "base64value"
+  const parseRfc9421Signature = (sigStr) => {
+    if (!sigStr) return sigStr
+    const match = sigStr.match(/^[^=]+=:([^:]+):$/)
+    return match ? match[1] : sigStr
+  }
+
+  // Convert standard base64 to base64url (HyperBEAM uses b64fast which expects base64url)
+  const toBase64url = (base64) => {
+    return base64
+      .replace(/\+/g, '-')
+      .replace(/\//g, '_')
+      .replace(/=/g, '')
+  }
+
+  // Extract and convert signature from standard base64 to base64url
+  const rawSignatureBase64 = parseRfc9421Signature(msg.headers.signature)
+  const rawSignature = toBase64url(rawSignatureBase64)
+
+  // Build 'committed' field listing signed components (required by HyperBEAM)
+  // Strip @ prefix from all derived components - HyperBEAM adds them back via add_derived_specifiers
+  const committedFields = components.map(v => v.startsWith('@') ? v.slice(1) : v)
+
+  // Get the keyid - must match EXACTLY what's in the signature-input
+  // send.js now includes the "publickey:" prefix in the keyid
+  // The commitment's keyid must be identical for verification to work
+  const keyid = sigParams.keyid || `publickey:${pub.toString('base64')}`
+
+  // RSA commitment (with committer)
+  // HyperBEAM expects: type, keyid, signature, committed, commitment-device, committer
+  // The keyid must match exactly what was in the signature-input for verification
+  const rsaCommitment = {
+    "commitment-device": "httpsig@1.0",
+    type: "rsa-pss-sha512",
+    keyid: keyid,
+    committer: committer,
+    signature: rawSignature,
+    committed: committedFields,
+  }
+
+  // Add tag if present in signature params (hashpath)
+  if (sigParams.tag) {
+    rsaCommitment.tag = sigParams.tag
   }
+
+  // HMAC commitment (no committer, keyid is "constant:ao")
+  // For HMAC, the signature IS the HMAC ID (they're the same value)
+  // This is because HMAC is deterministic - can be recomputed from message content
+  const hmacCommitment = {
+    "commitment-device": "httpsig@1.0",
+    type: "hmac-sha256",
+    keyid: "constant:ao",
+    signature: hmacId,
+    committed: committedFields,
+  }
+
   const committed = {
     commitments: {
-      [rsaId]: { ...meta, committer, ...sigs },
-      [hmacId]: { ...meta2, ...sigs },
+      [rsaId]: rsaCommitment,
+      [hmacId]: hmacCommitment,
     },
     ...body,
   }

hbsig/src/httpsig.js

@@ -142,8 +142,10 @@ function groupIds(map) {
 
   for (const [key, value] of Object.entries(map)) {
     if (isId(key) && typeof value === "string") {
-      // Store with lowercase key as in Erlang
-      idDict[key.toLowerCase()] = value
+      // FIX: Preserve case for commitment IDs (base64url is case-sensitive)
+      // Previous code lowercased keys, but HyperBEAM's commitment IDs like
+      // "3Dmi2IVTNFi-c7hjsuEr5D3kLNRwvLXFdj4REIzdwT4" must keep their case
+      idDict[key] = value
     } else {
       stripped[key] = value
     }
@@ -157,8 +159,7 @@ function groupIds(map) {
     stripped["ao-ids"] = items.join(", ")
   }
 
-  // Return IDs as lowercase in the result since they will be normalized
-  // when processed as headers
+  // Return IDs preserving their original case (base64url is case-sensitive)
   for (const [k, v] of Object.entries(idDict)) {
     stripped[k] = v
   }
@@ -200,8 +201,9 @@ function groupMaps(map, parent = "", top = {}) {
   const entries = Object.entries(map).sort(([a], [b]) => a.localeCompare(b))
 
   for (const [key, value] of entries) {
-    // Normalize keys to lowercase
-    const normKey = normalizeKey(key)
+    // FIX: Preserve case for ID keys (43-char base64url strings like commitment IDs)
+    // HTTP header names should be normalized to lowercase, but base64url IDs are case-sensitive
+    const normKey = isId(key) ? key : normalizeKey(key)
     const flatK = parent ? `${parent}/${normKey}` : normKey
 
     if (