MinerTxJob.update_timestamp can stamp a tx with its own parent's timestamp, producing txs the fullnode rejects

[tuliomir]

Summary

When two transactions are submitted within the same wall-clock second and one is selected as the other's parent, the service stamps the child with the same timestamp as its parent and the fullnode deterministically rejects the mined result:

full validation failed: tx=263fb774… timestamp=1785951640, parent=051a87f8… timestamp=1785951640

The fullnode requires tx.timestamp > parent.timestamp (1-second granularity), but the service assigns parents and timestamp independently: parents are fetched as bare hashes, and the timestamp is always int(time()). The client cannot prevent or repair this — both fields are assigned inside the service, under the nonce — so its only recourse is to re-mine and resubmit.

This surfaced while running wallet-lib's integration suite with parallel jest workers (all txs flow through one tx-mining-service): a quiet DAG makes the newest tx almost always a tip, so any two same-second broadcasts can collide. The same mechanics apply to any two wallets sharing a service instance in the same second.

Proposed fix

Make the stamped timestamp respect the parents:

tx.timestamp = max(int(txstratum.time.time()), max_parent_timestamp + 1)

This stays within consensus rules: hathor-core accepts timestamps up to MAX_FUTURE_TIMESTAMP_ALLOWED = 300 seconds in the future, and the clamp exceeds "now" by at most 1 second per same-second parent chain.

Suggested rollout: DevMiner first

The DevMiner is the variant integration tests use, and it stamps the timestamp in exactly one place (txstratum/dev/manager.py, right before solve_tx). Landing the fix there first lets every client that runs integration tests against it validate the behavior thoroughly under real parallel load, at zero risk to production. Once proven, apply the same rule to the production path (MinerTxJob.update_timestamp), which also refreshes the timestamp every JOB_UPDATE_INTERVAL = 2 seconds during mining.

The one design decision is how the service learns parent timestamps, since backend.get_tx_parents() returns hashes only:

1. Fetch each parent tx from the fullnode after resolving parents (two extra reads per job, cacheable) — single-repo change; or
2. Extend the fullnode's parents endpoint to return (hash, timestamp) pairs — cleaner, but cross-repo.

Option 1 seems sufficient to start, and the block path already has a TODO pointing the same direction (consuming timestamp bounds from the newer mining API).

---

Annex — details for whoever picks this up

Everything below is reference material gathered while diagnosing; nothing here changes the proposal above.

Code pointers (current master):

• txstratum/jobs.py — MinerTxJob.update_timestamp(): self._tx.timestamp = int(txstratum.time.time()), unconditional. Called on job creation and refreshed by the protocol loop (txstratum/protocol.py, JOB_UPDATE_INTERVAL = 2).
• txstratum/dev/manager.py — DevMiner path: tx.timestamp = int(txstratum.time.time()) immediately before solve_tx, single call site, already carries a comment about the fullnode's timestamp-delta validation.
• txstratum/manager.py — add_parents(): parents: List[bytes] = await self.backend.get_tx_parents() → job.set_parents(parents). Hashes only; parent timestamps are never known to the service.
• txstratum/api.py — MAX_TIMESTAMP_DELTA: int = 300 (service-side guard, mirrors the fullnode's tolerance).
• hathor-core: MAX_FUTURE_TIMESTAMP_ALLOWED = 5 * 60 (hathorlib/conf/settings.py); enforced in vertex_handler.py. The tx.timestamp > parent.timestamp requirement is what produces the full validation failed above.

Why a blind now + 1 is not enough: a same-second chain defeats it — tx A stamped T+1 (parent at T) can itself be picked as parent by tx B submitted in the same second, which would also stamp T+1. The clamp must read the actual parent timestamps.

Observed frequency (wallet-lib integration suite on a privnet, ~800 broadcasts per full run, one shared tx-mining-service, AVG_TIME_BETWEEN_BLOCKS: 1):

• 2 parallel jest workers: 6 rejections per ~25 min run.
• 4 parallel jest workers: 73 rejections in ~14 min — enough to exhaust test-level retries and fail suites.
• Forensic pairing of every rejection confirmed the parent always came from a different OS process (another jest worker, or the integration-test-helper's own funding transaction — the helper is equally affected as a client).

Client-side workaround in place: wallet-lib's integration setup now re-mines and resubmits on this specific rejection (signatures cover only the funds part, so the same signed tx can be re-mined with fresh parents/timestamp). That unblocks parallel test execution but leaves every other client of the service exposed, which is why this issue proposes fixing it at the source.

[tuliomir]

Implementation update: the parents are only half the invariant, and the other half does not scale

While implementing the clamp on the DevMiner path, it turned out that satisfying the parents rule is not sufficient. hathor-core enforces two separate timestamp rules, in two different verifiers, and both must hold:

Rule	Enforced in	Meaning
tx.timestamp > parent.timestamp	hathor/verification/vertex_verifier.py:92	must beat its DAG parents
tx.timestamp > spent_tx.timestamp	hathor/verification/transaction_verifier.py:154	must beat every tx it spends

Neither set contains the other. Parents are drawn from the current tips, while a spent tx may have been confirmed long ago and can therefore never be selected as a parent. A wallet spending an output of a transaction broadcast moments ago hits the second rule with the first one perfectly satisfied.

The naive extension of option 1 from the issue body — fetch each predecessor's timestamp — therefore has to cover the inputs too, and that is where it stops scaling. A transaction can carry 200+ inputs, and hathor-core rate-limits the endpoint we would be calling:

• /v1a/transaction — 50 r/s, burst 100 (hathor/transaction/resources/transaction.py:322)
• /v1a/tx_parents — 30 r/s (hathor/transaction/resources/tx_parents.py)

At 200 inputs and 10 tx/s that is 2000 r/s against a 50 r/s budget. Worse, it is self-defeating: every throttled lookup degrades to "timestamp unknown", i.e. no clamp at all, so the mechanism fails hardest under exactly the concurrency that motivated this issue.

One approach worth ruling out early

The tempting shortcut is a single global bound: ask the fullnode for the newest timestamp in the DAG, U, and stamp max(now, U + 1). It is one request, and since every edge enforces child.timestamp > parent.timestamp, U provably dominates every vertex.

It ratchets. Every tx we mine becomes the new U, so at 10 tx/s the stamps go 1, 2, … 10 within the first wall-clock second and the lead over now grows about 9 s per second, reaching MAX_FUTURE_TIMESTAMP_ALLOWED = 300 in roughly 33 seconds of sustained load. Unrelated transactions interact, which per-predecessor clamping never does. The conclusion is that the per-predecessor model is structurally right and the work should go into making the lookups cheap, not into replacing them with a coarser bound.

Options

A. Have the client supply the floor, and stop discarding it. The wallet chooses the inputs, so it is the party that already knows their timestamps — it selects them from its own UTXO set and the spent transactions are in its local storage. In wallet-lib this is IStorage.getSpentTxs(inputs) (src/types.ts:817) plus IHistoryTx.timestamp (src/types.ts:269): an accessor that already exists, computable with zero network I/O. The service would then only clamp against the parents, which it chose itself — at most two hashes, regardless of input count.

Two things currently stand in the way. txstratum/api.py:179 accepts the client's timestamp but the DevMiner path overwrites it (_get_timestamp_for returns max(now, preds + 1) and never considers tx.timestamp), and in production MinerTxJob.update_timestamp() sets plain now on every job dispatch (txstratum/protocol.py:575). Wallet-lib also unconditionally adopts the value the service returns (src/new/sendTransaction.ts:1105) — correctly, since the nonce is solved over the service's timestamp, but it means the client's value is discarded end to end today.

This is the option that matches how responsibility is actually divided: whoever chooses a predecessor owns the constraint against it. It needs a coordinated wallet-lib change, so it is the destination rather than the immediate step.

B. Remember our own output. After solving, record hash → timestamp for every tx this service mines. A chained flock arriving at one service is, by construction, mostly predecessors that this same service produced moments earlier, so those lookups become memory reads. Roughly four lines, no cross-repo dependency, and it covers precisely the case the issue's forensics identified (parallel jest workers, one shared service).

C. Prove the lookups unnecessary with a single request. GET /v1a/dashboard_tx?block=1&tx=1 returns the newest block and the newest tx in one call — blocks matter because an input can spend a mining reward. Call the resulting maximum U. If now > U, then no predecessor can possibly be at or above now, so now is valid and every per-input lookup can be skipped. Note the difference from the ruled-out approach above: U is used as a test, never as a floor, so it cannot ratchet. It cannot be cached, since a stale U under-estimates and would be unsafe, but one request replacing two hundred is the point. When the test fails, fall back to per-predecessor lookups — which is where B is already covering the hot path.

D. Move it into the fullnode. This is option 2 from the issue body, generalised. Extending tx_parents to return (hash, timestamp) pairs zeroes out the parents lookup, but on its own it does nothing for the inputs, which is now the expensive half. The version that solves the whole problem is a single endpoint that takes the predecessors and returns the answer — e.g. POST /v1a/min_timestamp {parents, inputs} — computed in-process by the node with no per-predecessor HTTP at all. One request, exact, no ratchet. Cleanest of the four and the slowest to land.

Suggested sequencing

B and C are entirely inside this repository, need no wallet or fullnode change, and together take the flock scenario from thousands of requests per second to approximately zero: C removes the per-input lookups whenever the DAG is quiet, and B serves them from memory when it is not. A is where the design should end up and is worth opening on the wallet-lib side in parallel, since its value is independent — a wallet pushing directly to a fullnode gets the fix immediately, without any service involvement. D remains the cleanest endpoint and can supersede C later.