Skip to content

Htlc preparation - #65

Draft
sbn20241 wants to merge 29 commits into
devfrom
htlc_preparation
Draft

sbn20241 wants to merge 29 commits into
devfrom
htlc_preparation

Conversation

@sbn20241

@sbn20241 sbn20241 commented Jul 24, 2026

Copy link
Copy Markdown
Collaborator

We needed to close the RGB HTLC flow.

The lock is a foreign HTLC UTXO, not an rgb-lib receive. witness_receive is only the claim/refund destination — a normal wallet-owned output on the colored keychain. rgb-lib does not register the HTLC script as a receive; RGB on the lock is inspected with contract_assignments_for_outpoints. After the claim, the receiver imports onto the witness_receive output.

The claim/refund PSBT is built externally (native SegWit inputs, unsigned). Coloring must happen in rgb-lib, but consume_fascia is not reversible, so this is two-phase: psbt_op_prepare colors the PSBT, writes the fascia and the per-asset consignments under operation_dir and registers an Initiated batch with the witness txid and an expiration (stash unchanged); after broadcast, once the indexer sees the tx, psbt_op_apply consumes the fascia and moves the batch to WaitingConfirmations. If the tx never lands, psbt_op_abort, or the expiry sweep fails the batch. psbt_op_reconcile reports the operation status. fail_transfers refuses to release the inputs of a prepare batch whose tx the indexer already knows; send_begin batches are unaffected.

Upload the consignment from operation_dir/consignments/ to the RGB proxy; the receiver imports with fetch_and_accept_transfer_by_recipient_id (pins output[vout] to the witness script, checks confirmations, compares the received assignment with ExpectedTransfer).

These were rust-only and unreachable from Go. Uniffi/UDL now exports psbt_op_prepare / psbt_op_apply / psbt_op_abort / psbt_op_reconcile, contract_assignments_for_outpoints, fetch_and_accept_transfer_by_recipient_id and script_hex_from_recipient_id.

issue

PR and solution are based on @txalkan PRs #4 & #5, adapted to prepare → broadcast → apply instead of coloring and consuming fascia in one call.

@sbn20241
sbn20241 marked this pull request as ready for review August 3, 2026 22:37
@txalkan

txalkan commented Aug 4, 2026

Copy link
Copy Markdown
Member

Hey guys, it seems that rgb-lib intentionally does not support arbitrary script receives as an architectural boundary — or at least that’s what I gathered from the context back in February. See the second page of this file for more details:

feat_submarine_swap_v0.3.pdf

An escrow solution was implemented in txalkan/rgb-lightning-node@157190a — with support from these rgb-lib PRs: UTEXO-Protocol/rgb-lib-old#4 & UTEXO-Protocol/rgb-lib-old#5

@gofman8

gofman8 commented Aug 4, 2026

Copy link
Copy Markdown

@txalkan is right here:
some more context RGB-Tools#77

@sbn20241
sbn20241 marked this pull request as draft August 5, 2026 00:04
mirvaisdia-nitka and others added 5 commits August 5, 2026 02:18
CI failed because security_gaps/shared_cosigner were untracked and inspect_rgb_transfer had an unclosed function.

Co-authored-by: Cursor <cursoragent@cursor.com>
@claude

claude Bot commented Aug 18, 2026

Copy link
Copy Markdown

Review: HTLC preparation

Exposing color_psbt_and_consume over UniFFI is the right shape for this flow, and the is_subset guard on color_psbt_for_outpoints shows the right instinct. Two categories of concern below: the branch does not currently build, and there are correctness risks specific to feeding externally constructed PSBTs into color_psbt.

Blocking - build/lint failures

1. src/wallet/test/multisig/mod.rs declares modules whose files are not in the PR. It adds mod security_gaps; and mod shared_cosigner; (both gated on the electrum feature), but src/wallet/test/multisig/ contains only mod.rs and utils.rs, and neither file appears in the diff. That is an unresolved-module error, so cargo clippy --lib --workspace --all-features --all-targets -- -D warnings (lint.yml) and cargo test both fail. Add the files or drop the declarations.

2. Once (1) is fixed, the new helpers in multisig/utils.rs become dead code. psbt_signed_by_cosigner and psbt_with_foreign_tap_script_sigs are pub(super) with no callers besides the missing modules; -D warnings makes dead_code a hard error.

3. Debris in src/wallet/test/mod.rs: the commented-out mod swaply_htlc_lock; line with its "missing on this branch" note should not land. Same root cause as (1) - branch-local files were pruned and these were missed.

4. cargo fmt --check will fail. In bindings/uniffi/src/lib.rs the bdk_wallet::bitcoin::{...} and rust_only::{...} use-groups both fit on one line and rustfmt will collapse them, and the accept_transfer_from_consignment(...) call args will be joined. In multisig/utils.rs the psbt_with_foreign_tap_script_sigs signature is 103 chars, over the 100-col limit.

Correctness / safety

5. color_psbt_for_outpoints can silently destroy allocations on the excluded inputs. The subset check confirms the override outpoints are in the PSBT, but nothing checks the complement. prev_outputs drives contract_assignments_for(...) -> add_input(opout, state), so any RGB allocation on a PSBT input the caller omitted from input_outpoints is never added to the transition - yet its seal is closed when the tx confirms. That allocation is lost. color_psbt derived prev_outputs from all inputs precisely so this could not happen; the new entry point removes that net and exposes it to Go. Per CLAUDE.md (UTXO/accounting invariants, explicit errors over silent fallback), I would query contract_assignments_for over psbt_inputs - override_set and return InvalidColoringInfo if non-empty - ideally across all known contracts, not just the ones being colored, since an unrelated asset sharing a spent UTXO is the likelier footgun.

6. An externally built PSBT loses its input metadata. src/wallet/rust_only.rs:396: *psbt = Psbt::from_unsigned_tx(transaction).unwrap();. When the incoming PSBT has no OP_RETURN, color_psbt appends one and rebuilds from the unsigned tx - and from_unsigned_tx creates empty input/output maps, discarding witness_utxo, tap_internal_key, tap_scripts, tap_merkle_root, partial_sigs and bip32 derivations. Harmless for the existing internal flow (rgb-lib built the PSBT and re-derives everything), but this PR is explicitly about a claim/refund PSBT "constructed externally": such a P2TR script-path PSBT comes back unsignable. Also, when any output is P2TR, opreturn_first inserts the OP_RETURN at index 0, shifting every output of the caller tx. output_map compensates internally (vout += 1), but the caller no longer has the output layout it built - which matters if the HTLC leaf or a counterparty pre-signature commits to it. Either preserve the maps when rebuilding, or require the caller to supply the OP_RETURN in the expected position and reject PSBTs that do not.

7. Reachable panic across the FFI boundary. Same line - .unwrap(). from_unsigned_tx errors if any input carries a script_sig or witness, and extract_tx() above populates those from the final_script_* fields. A finalized or partly finalized PSBT - now trivially passable from Go - panics instead of erroring. src/wallet/mpc_psbt.rs:64 already handles this with map_err(|e| Error::Internal {..}); worth matching.

8. script_hex_from_recipient_id conflates "blinded" with "invalid". script_buf_from_recipient_id returns Ok(None) for a valid Beneficiary::BlindedSeal. Mapping that to InvalidRecipientID tells the caller their input was malformed when it was not - suggest string? in the UDL or a distinct error.

9. Wrong error variant for a bad asset id. contract_assignments_for_outpoints returns InvalidColoringInfo when ContractId::from_str fails, but the method takes no coloring info. export_asset_contract uses Error::Internal { details: format!("invalid asset ID: {error}") } for the same case.

Quality

10. color_psbt_for_outpoints_and_consume is a verbatim copy of color_psbt_and_consume apart from the first call. Having already extracted color_psbt_with_prevouts, the follow-through is a private consume_and_transfer(psbt, fascia, beneficiaries) shared by both - ~40 duplicated lines of seal-splitting logic is exactly what drifts.

11. Same in the bindings: you added save_rgb_transfer but both color methods inline the identical transfer.save(&mut buf) loop instead of calling it. Also coloring_info.clone() is unused after the call.

12. accept_transfer logs "Accepting transfer..." before the split point, so the new accept_transfer_from_consignment emits "Accept transfer completed" with no matching start line.

13. contract_assignments_for_outpoints omits outpoints with no state, so a Go caller cannot tell "no allocations" from "not in the result". Returning an empty assignments vec per requested outpoint would suit an HTLC resolver checking whether the lock is funded.

Scope

14. The description leads with script_witness_receive, but no such function is anywhere in the diff (grep -ri htlc src/ bindings/ finds nothing outside the new doc comments). Please update it so reviewers know what they are looking at.

15. The multisig security-gap helpers and inspect_rgb_transfer_does_not_validate_psbt_signatures are unrelated to HTLC preparation. The latter is also a characterization test asserting a missing safety property - if signature validation is ever added, it fails and the fix is to delete it. Better tracked as an issue; if it stays, please split it out.

Test coverage

16. None of the new entry points has a test - not color_psbt_for_outpoints, color_psbt_for_outpoints_and_consume, accept_transfer_from_consignment, contract_assignments_for_outpoints, fetch_consignment_by_recipient_id, or script_hex_from_recipient_id. At minimum: the two error paths in color_psbt_for_outpoints (empty and non-subset, both already written); the asset-loss case from (5) - spend two allocated UTXOs, list only one, assert the balance afterward, which would either confirm the concern or prove me wrong; a round trip on a PSBT that already has an OP_RETURN showing input metadata survives (6); and script_hex_from_recipient_id for both witness and blinded recipient ids.

Summary

The FFI plumbing itself is straightforward. (1)-(4) are blocking - the branch does not build or lint. The ones I would most want resolved before merge are (5) and (6): both concern color_psbt being handed a PSBT it did not construct, which is the new capability here, and both lose assets rather than erroring. Happy to re-review once the missing files and CI are sorted.

mirvaisdia-nitka and others added 2 commits August 18, 2026 15:32
Co-authored-by: Cursor <cursoragent@cursor.com>
Parties only hold references, so end their borrows with a block before dropping wallets.

Co-authored-by: Cursor <cursoragent@cursor.com>
@github-actions

Copy link
Copy Markdown

🌗 Pull Request Overview

This PR introduces HTLC (submarine swap) preparation features for RGB asset flows. It adds new Rust-only wallet methods for coloring PSBTs with explicit outpoint sets, fetching/accepting RGB consignments by recipient ID, and inspecting contract assignments on arbitrary outpoints. It also adds UniFFI bindings for these methods and includes regression tests documenting known multisig PSBT validation gaps and shared-cosigner wallet isolation.

Reviewed Changes
Kimi performed full review on 9 changed files and found 3 issues.

Show a summary per file
File Description
bindings/uniffi/src/lib.rs Added FFI structs (ColoringInfo, ColorPsbtResult, etc.), helper functions (to_rgb_coloring_info, save_rgb_transfer, etc.), and 5 new Wallet methods for HTLC flows. Also added script_hex_from_recipient_id top-level function.
bindings/uniffi/src/rgb-lib.udl Added UDL definitions for new FFI dictionaries (AssetColoringInfo, ColoringInfo, ColorPsbtResult, FetchConsignmentResult, AcceptTransferResult, OutpointAssignments) and new Wallet method signatures.
src/wallet/rust_only.rs Refactored color_psbt to delegate to new color_psbt_with_prevouts. Added color_psbt_for_outpoints, color_psbt_for_outpoints_and_consume, fetch_consignment_by_recipient_id, accept_transfer_from_consignment, accept_transfer_with_consignment, and contract_assignments_for_outpoints.
src/wallet/test/mod.rs Added commented-out module reference (swaply_htlc_lock).
src/wallet/test/multisig/mod.rs Added module declarations for security_gaps and shared_cosigner (gated by feature = "electrum").
src/wallet/test/multisig/security_gaps.rs New test documenting that inspect_psbt and respond_to_operation do not validate foreign cosigner tap_script_sigs.
src/wallet/test/multisig/shared_cosigner.rs New tests verifying offline descriptor isolation and full hub e2e for multisig wallets sharing one cosigner key.
src/wallet/test/multisig/utils.rs Added psbt_signed_by_cosigner and psbt_with_foreign_tap_script_sigs test helpers.
src/wallet/test/rust_only.rs Added test documenting that inspect_rgb_transfer accepts unsigned PSBTs.

📋 Review Findings

📄 bindings/uniffi/src/lib.rs

🟡 MEDIUM logic: Duplicate asset_id entries silently overwritten in to_rgb_coloring_info

Lines 175–186

The conversion iterates over coloring_info.assets and inserts each into a HashMap keyed by ContractId. If the caller accidentally provides the same asset_id twice (e.g., two AssetColoringInfo entries with the same ID but different output_maps), the second entry silently overwrites the first. This can lead to unexpected coloring behavior where one asset's instructions are lost.

💡 Suggested fix:

Current code:

fn to_rgb_coloring_info(coloring_info: ColoringInfo) -> Result<RgbColoringInfo, RgbLibError> {
    let mut asset_info_map = HashMap::new();
    for asset in coloring_info.assets {
        let contract_id =
            ContractId::from_str(&asset.asset_id).map_err(|e| RgbLibError::InvalidColoringInfo {
                details: format!("invalid asset_id '{}': {e}", asset.asset_id),
            })?;
        asset_info_map.insert(
            contract_id,
            RgbAssetColoringInfo {
                output_map: asset.output_map,
                static_blinding: asset.static_blinding,
            },
        );
    }
    ...
}

Improved code:

fn to_rgb_coloring_info(coloring_info: ColoringInfo) -> Result<RgbColoringInfo, RgbLibError> {
    let mut asset_info_map = HashMap::new();
    for asset in coloring_info.assets {
        let contract_id =
            ContractId::from_str(&asset.asset_id).map_err(|e| RgbLibError::InvalidColoringInfo {
                details: format!("invalid asset_id '{}': {e}", asset.asset_id),
            })?;
        if asset_info_map.contains_key(&contract_id) {
            return Err(RgbLibError::InvalidColoringInfo {
                details: format!("duplicate asset_id '{}'", asset.asset_id),
            });
        }
        asset_info_map.insert(
            contract_id,
            RgbAssetColoringInfo {
                output_map: asset.output_map,
                static_blinding: asset.static_blinding,
            },
        );
    }
    ...
}

🔵 LOW api ergonomics: Misleading error variant for invalid asset_id in contract_assignments_for_outpoints

Lines 1317–1319

When parsing the direct asset_id parameter fails, the code returns RgbLibError::InvalidColoringInfo. This is semantically confusing for API consumers because no ColoringInfo struct is involved in this call.

Current code:

let contract_id =
    ContractId::from_str(&asset_id).map_err(|e| RgbLibError::InvalidColoringInfo {
        details: format!("invalid asset_id '{asset_id}': {e}"),
    })?;

If RgbLibError has (or can have) a more generic variant such as InvalidAssetID, prefer that. Otherwise, consider adding one to avoid misleading callers.


🔵 LOW maintainability: save_rgb_transfer helper exists but is not used

Lines 1260–1275 and 1293–1308

Both color_psbt_and_consume and color_psbt_for_outpoints_and_consume inline the exact same consignment-serialization logic that the save_rgb_transfer helper (line 206) already provides. Using the helper reduces duplication and prevents the inline error messages from diverging.

Current code (in both methods):

let mut consignments = Vec::with_capacity(transfers.len());
for transfer in &transfers {
    let mut buf = Vec::new();
    transfer.save(&mut buf).map_err(|e| RgbLibError::Internal {
        details: format!("serialize consignment: {e}"),
    })?;
    consignments.push(buf);
}

Improved code:

let mut consignments = Vec::with_capacity(transfers.len());
for transfer in &transfers {
    consignments.push(save_rgb_transfer(transfer)?);
}

📄 bindings/uniffi/src/rgb-lib.udl

No issues found. The new dictionaries and method signatures map cleanly to the Rust FFI layer.


📄 src/wallet/rust_only.rs

No issues found. The refactoring of color_psbt into color_psbt_with_prevouts is clean, input validation in color_psbt_for_outpoints is correct, and the new HTLC helpers are appropriately gated with #[cfg(any(feature = "electrum", feature = "esplora"))].


📄 src/wallet/test/mod.rs

No issues found. The commented-out module reference is harmless documentation.


📄 src/wallet/test/multisig/mod.rs

No issues found. Module declarations are correctly feature-gated.


📄 src/wallet/test/multisig/security_gaps.rs

No issues found. The test accurately documents the existing validation gap with clear assertions and a descriptive comment.


📄 src/wallet/test/multisig/shared_cosigner.rs

No issues found. Tests cover offline descriptor differentiation, full e2e hub isolation, and duplicate-cosigner behavior. Directory isolation assertions are solid.


📄 src/wallet/test/multisig/utils.rs

No issues found. The psbt_with_foreign_tap_script_sigs helper is well-contained test infrastructure with appropriate assertions.


📄 src/wallet/test/rust_only.rs

No issues found. The test clearly documents the inspect_rgb_transfer signature-validation gap.


Powered by Kimi | Model: kimi-k2.6

@UTEXO-Protocol UTEXO-Protocol deleted a comment from github-actions Bot Aug 18, 2026
@sbn20241
sbn20241 marked this pull request as ready for review August 18, 2026 21:58
@sbn20241
sbn20241 requested review from gofman8 and txalkan August 18, 2026 22:00
@gofman8

gofman8 commented Aug 18, 2026

Copy link
Copy Markdown

@mirvaisdia-nitka I still have my comments unaddressed and AI report as well

@sbn20241
sbn20241 force-pushed the htlc_preparation branch 2 times, most recently from c8bc87f to 4c8fd5d Compare August 28, 2026 02:31
@sbn20241
sbn20241 marked this pull request as ready for review August 28, 2026 09:55
@sbn20241
sbn20241 marked this pull request as draft August 28, 2026 10:25

@Jainakin Jainakin left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I re-reviewed the current head 4c8fd5d6, all changed files, every existing review thread, and the related receive, accounting, backup and persistence paths.

The PSBT map preservation, omitted-allocation rejection, witness pinning, path validation, broadcast-before-apply guard and durable marker are meaningful improvements. I am still requesting changes because the operation is not crash-consistent across RGB stock, operation files, SQLite and VSS.

The blocking issues are:

  1. Stock is persisted and the runtime lock released before recovery evidence is written.
  2. Required payload files are not durably published or integrity-bound.
  3. Abort and bulk expiry can split SQL, metadata and VSS state.
  4. The required witness_receive accounting path is not implemented or tested end to end.
  5. External broadcasting leaves an ambiguous rollback window.
  6. A committed operation cannot be adopted after response loss.
  7. The new UniFFI path exposes a caller-triggerable UDA panic.

Before merge, the PR also needs Windows-safe payload paths, fail-closed handling for missing PSBTs, operation retention/pruning, removal of serial indexer calls from bulk failure, and explicit documentation of the public API compatibility changes.

Local formatting, Clippy, no-default-feature tests and UniFFI tests pass, and the GitHub matrix is green. I also built a targeted witness_receive integration test, but the repository Docker harness stalled while starting services, so that end-to-end path remains unverified.

Given the size and upstream design guidance, I recommend separating PSBT safety primitives, the durable HTLC lifecycle, and bindings into independently reviewable changes.

Comment thread src/wallet/rust_only.rs
Comment thread src/wallet/rust_only.rs
Comment thread src/wallet/rust_only.rs Outdated
Comment thread src/wallet/rust_only.rs Outdated
Comment thread src/wallet/rust_only.rs
Ok(())
}

/// Abort a prepared HTLC operation before apply (tx never broadcast).

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Never broadcast cannot be established from a single current indexer lookup. Broadcast response loss, indexer lag, mempool eviction or later rebroadcast can make the transaction appear absent during abort and visible afterward, after inputs have already been credited back. Please persist an explicit broadcast lifecycle such as NotAttempted, Attempted, Observed and Ambiguous, and prohibit automatic rollback for attempted or ambiguous operations. Alternatively, make broadcasting part of the library-owned operation.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed — one indexer lookup can’t prove the tx was never broadcast. Lag, lost broadcast response, mempool drop, or a later rebroadcast can make it missing at abort and show up after we’ve already rolled the inputs back.

Broadcast stays with the caller in this PR; htlc_abort is only for the “we never sent it” case, and that’s a caller promise, not something the library can verify yet.

Follow-up: either an explicit broadcast lifecycle (NotAttempted / Attempted / Observed / Ambiguous) and no automatic rollback once it’s been attempted or is ambiguous, or the library owns broadcast. Not in this PR.

@Jainakin Jainakin Sep 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review update at 0f30018: Still open, with a narrower requested fix. I accept the documented never-broadcast caller precondition for explicit abort and am not asking for a general broadcast journal here. Automatic expiry is different: an externally broadcast transaction may still have a Prepared/Initiated operation until apply completes. Expiry plus a temporary indexer not-found can fail/release it, after which apply rejects the Failed state when the transaction becomes visible. Please prevent automatic failure/release of these externally managed prepared operations based solely on expiry and current transaction absence. Making the expiry explicit does not establish that broadcasting never occurred.

Consolidated scope correction.

Earlier assessment at b213f76 (superseded where noted above)

Thanks for making the caller-promise limitation explicit. There is still an automatic path that bypasses that promise at b213f76: psbt_op_prepare defaults to a 24-hour expiration, and the bulk expiry sweep fails an Initiated prepare batch when get_tx_confirmations returns None. It has no durable evidence that the caller never attempted broadcast. A transaction can have been broadcast, become temporarily absent, then reappear after the batch was failed and allocations released; apply then refuses the Failed batch.

Indexer errors now stay fail-closed, which is good, but a successful 'not found' response is not the same as NotAttempted. Please introduce durable pre-broadcast handoff/attempt evidence and quarantine ambiguous attempts, or remove automatic expiry/release for externally broadcast operations until that lifecycle exists. Cover response loss, indexer lag and eviction/reappearance. Keeping broadcast with the caller is fine; interpreting absence as proof of never-broadcast is the unsafe part.

Comment thread src/wallet/rust_only.rs
Comment thread src/wallet/rust_only.rs
Comment thread src/wallet/rust_only.rs
Comment thread src/wallet/rust_only.rs
@sbn20241
sbn20241 force-pushed the htlc_preparation branch 2 times, most recently from 58fa4eb to 7aa2540 Compare August 28, 2026 16:44
@sbn20241

sbn20241 commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator Author

I re-reviewed the current head 4c8fd5d6, all changed files, every existing review thread, and the related receive, accounting, backup and persistence paths.

The PSBT map preservation, omitted-allocation rejection, witness pinning, path validation, broadcast-before-apply guard and durable marker are meaningful improvements. I am still requesting changes because the operation is not crash-consistent across RGB stock, operation files, SQLite and VSS.

The blocking issues are:

  1. Stock is persisted and the runtime lock released before recovery evidence is written.
  2. Required payload files are not durably published or integrity-bound.
  3. Abort and bulk expiry can split SQL, metadata and VSS state.
  4. The required witness_receive accounting path is not implemented or tested end to end.
  5. External broadcasting leaves an ambiguous rollback window.
  6. A committed operation cannot be adopted after response loss.
  7. The new UniFFI path exposes a caller-triggerable UDA panic.

Before merge, the PR also needs Windows-safe payload paths, fail-closed handling for missing PSBTs, operation retention/pruning, removal of serial indexer calls from bulk failure, and explicit documentation of the public API compatibility changes.

Local formatting, Clippy, no-default-feature tests and UniFFI tests pass, and the GitHub matrix is green. I also built a targeted witness_receive integration test, but the repository Docker harness stalled while starting services, so that end-to-end path remains unverified.

Given the size and upstream design guidance, I recommend separating PSBT safety primitives, the durable HTLC lifecycle, and bindings into independently reviewable changes.

@sbn20241
sbn20241 requested a review from Jainakin September 4, 2026 16:28
@Jainakin

Jainakin commented Sep 7, 2026

Copy link
Copy Markdown

I completed the requested review of the separated follow-up in #94 at exact head 201f3e00: #94 (review)

#94 does close several findings from my earlier review: required payloads are hash-bound and durably replaced, consignment paths are Windows-safe, missing PSBTs fail closed, the UDA panic is now a typed error, and the foreign-escrow witness_receive path has focused end-to-end coverage.

The crash-consistency review on this PR must remain open, however. The separated patch still allows generic fail_transfers to fail a broadcast-attempted HTLC (reproduced with a deterministic regression), does not expose broadcast/adoption through UniFFI, cannot adopt a lost prepare response using a txid the caller never received, relies on non-atomic/non-fsynced RGB stock persistence, has an orphan window between metadata and SQL publication, silently drops corrupt operation metadata, and does not make broadcast evidence VSS-durable. Retention/pruning and serial bulk indexer calls also remain.

The exact-head formatting and no-default-features Clippy gates currently fail, and no GitHub checks are registered for #94's stacked base. I left the actionable findings on #94 so this already-large PR does not accumulate another implementation layer.

mirvaisdia-nitka and others added 3 commits September 8, 2026 01:09
The prepare/apply/abort lifecycle is not HTLC-specific: any externally built
PSBT (timelock escrows, other scripts) goes through it. Rename the public API,
FFI surface, on-disk layout (psbt_ops/, foreign_inputs.json) and tests.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Without an expiration a prepared operation could keep its inputs reserved
forever; apply PSBT_OP_DEFAULT_EXPIRATION_SECS when the caller passes none.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@sbn20241
sbn20241 force-pushed the htlc_preparation branch 5 times, most recently from 28aca7c to b213f76 Compare September 9, 2026 15:02

@Jainakin Jainakin left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Scope correction at 0f30018: the merge requirements below have been narrowed after comparison with the UTEXO base and RGB-Tools upstream. See the updated assessment. Fixed findings are acknowledged there; deferred recovery work is not a blanket blocker for #65. The remaining change request concerns automatic expiry of externally broadcast prepared operations.

Historical review at b213f76 (superseded where noted above)

Re-reviewed at b213f76, including the update pushed during this review, the earlier human/AI reviews, and the separate recovery work in #94. I cannot approve this head for production yet.

There are real improvements here: foreign inputs are no longer invented as wallet TXOs; the normal witness_receive + refresh path has a single accounting owner; missing persisted PSBTs fail closed; the UDA unwrap is replaced with a typed error; legacy output-index semantics and PSBT maps are covered; and the latest update makes repeat apply succeed and validates native SegWit inputs and the recipient's colored keychain. I am not repeating those as unresolved implementation defects.

Remaining correctness/recovery gates

  1. Stock consumption, consumed markers, SQL and operation metadata still do not form a recoverable persistence boundary. The injected failure occurs after both markers, not between stock persistence and marker publication.
  2. An absent indexer transaction still permits releasing externally broadcast inputs. The default 24-hour expiration makes this an automatic path, not merely an explicit caller promise that broadcasting was never attempted.
  3. Required payloads are ordinary mutable writes without a versioned, integrity-bound manifest. Durable metadata is not sufficient to make the payloads durable or mutually consistent.
  4. Aborting now repairs an already-failed batch on an explicit retry, but reconciliation still returns Prepared for Prepared metadata paired with Failed SQL. Bulk expiry and the intermediate auto-backup snapshot can produce this state.
  5. A successful prepare whose response is lost still has no discover/adopt API. In the string-based UniFFI call the caller may never receive the colored PSBT or its final txid, so lookup by that final txid alone is not a complete recovery contract.
  6. Rejected duplicate preparation changes existing transfer artifacts, including the marker that decides which consume API owns the operation. See the new inline reproduction.
  7. The documented receive helper imports into RGB stock but does not itself finish the ordinary witness_receive SQL/balance/history lifecycle. The new accounting test deliberately uses refresh instead of this helper; the supported caller flow needs to be explicit and tested.
  8. Per-asset filenames still contain the colon from the RGB contract ID and fail on Windows.

The separate scope of #94 is reasonable, but it is still open and targets htlc_preparation; its changes are not in this head. These are therefore unresolved production gates for the combined feature, not fixes already delivered by #65. Please either stack the required corrections into the release candidate or keep this API explicitly unavailable for production until the dependent recovery work is integrated and tested. I am not asking to fold unrelated recovery architecture into this diff.

Verification

  • Current-head no-default-feature library tests: 56 passed.
  • Current-head UniFFI library tests: 5 passed; these are conversion/validation tests, not a generated-Go end-to-end test.
  • Formatting check passed.
  • Four isolated characterization probes reproduced the current behavior using disposable wallets: stale Failed/Prepared reconciliation, duplicate-prepare marker mutation, abort accepting an invalid Online handle when no linked batch exists, and malformed native Rust PSBT output maps panicking. The first two are directly relevant to recovery. The last two are smaller API-hardening items; the malformed-struct panic is not a claim that malformed serialized input gets through UniFFI's PSBT parser.
  • The duplicate-write probe uses the real persistence helper/SQLite/filesystem with a minimal fascia fixture. It is not presented as a funded HTLC round trip.
  • No new full Docker/Go/OS-kill/Windows runtime test was run locally in this review. Current-head GitHub integration CI was still running when checked; passing build jobs must not be treated as proof of these failure boundaries.

The inline comments and replies distinguish fresh findings from earlier threads that remain unresolved, and include the specific regression cases needed. No production files or dependency pins were changed for this review.

Comment thread src/wallet/rust_only.rs Outdated
Comment thread src/wallet/rust_only.rs
Comment thread src/wallet/test/rust_only.rs Outdated
@sbn20241
sbn20241 force-pushed the htlc_preparation branch 5 times, most recently from 99e9dbf to 0f30018 Compare September 10, 2026 10:10
@Jainakin

Copy link
Copy Markdown

Review scope correction at 0f30018

Re-reviewed following Renat's scope clarification, comparing this PR with both its UTEXO base and RGB-Tools upstream.

My earlier review mixed defects introduced by this PR with broader, inherited durability limitations. The merge requirements were too broad, and I am correcting that.

I am withdrawing whole-wallet atomic persistence, mandatory payload hashes, a complete operation-adoption framework, and a VSS redesign as blockers for #65. The general persistence limitations are inherited; the VSS behavior is inherited from the UTEXO fork. Lost-response adoption remains a follow-up for the new API, not a requirement to incorporate all of #94 into this PR. These items are deferred, not claimed fixed.

The latest changes address the duplicate-write, failed-status reconciliation, Windows filename, UDA test-fixture, and online-handle findings. The receive helper's stock-only behavior is consistent with the existing low-level API model; the required normal refresh step is now documented and has an integration test. The malformed native Rust PSBT output-map panic also follows an existing upstream pattern, so I am not treating that as a new blocker here.

Remaining request: automatic expiry of externally broadcast prepared operations.

In the new split workflow, a transaction can be successfully broadcast while its batch remains Initiated until apply completes. If the indexer temporarily returns not-found, the expiry sweep can fail that batch, and a later apply then rejects it when the transaction reappears. This is a supported broadcast-to-apply interval, not just an inherited crash-between-stores problem.

The requested fix is narrower than my earlier recovery proposal: prevent automatic failure/release of these externally managed prepared operations based solely on expiry and current transaction absence. Explicit abort can retain its documented caller precondition that broadcasting was never attempted. A general broadcast journal does not need to be introduced here. The relevant discussion is this thread.

Verification: I locally verified the repaired reconciliation, duplicate-artifact preservation in both ownership orders, and online-handle checks using disposable-wallet probes. These are not funded end-to-end tests. Current CI is still running, so I am not claiming the complete integration suite has passed.

This supersedes the broader merge requirements in my previous review. I am retaining the change request for the narrow automatic-expiry behavior above, not for a general rgb-lib recovery redesign.

Jainakin
Jainakin previously approved these changes Sep 10, 2026
Resolve UDL conflict (keep asset_schema_id and script_hex_from_recipient_id)
and adapt pin_witness_output_to_recipient_id to the new
ResolveWitness::resolve_witness(&PubWitness) signature.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants