v4.2.0

v4.2.0 🌿

Happy new year! We have an incredibly exciting release to kick off the new year. This release is strongly recommended for all operators to update as it has many bug fixes, security patches, and features that will improve the Prysm experience on mainnet. This release has so many wonderful changes that we've deviated from our normal release notes format to aptly categorize the changes.

Highlights

Upgrading / Downgrading Validators

There are some API changes bundled in this release that require you to upgrade or downgrade in particular order. If the validator is updated before the beacon node, it will see repeated 404 errors at start up until the beacon node is updated as it uses a new API endpoint introduced in v4.2.0.

🔼 Upgrading: Upgrade the beacon node, then the validator.
🔽 Downgrading: Downgrade the validator to v4.1.1 then downgrade the beacon node.

Deneb Goerli Support

This release adds in full support for the upcoming deneb hard fork on goerli next week on January 17th.

Networking Parameter Changes

This release increases the default peer count to 70 from 45. The reason this is done is so that node's running
with default peer counts can perform their validator duties as expected. Users who want to use the old peer count
can add in --p2p-max-peers=45 as a flag.

Profile Guided Optimization

This release has binaries built using PGO, for more information on how it works feel free to look here: https://tip.golang.org/doc/pgo .
This allows the go compiler to build more optimized Prysm binaries using production profiles and workloads.

ARM Supported Docker Images

Our docker images now support amd64 and arm64 architecture! This long awaited feature is finally here for Apple Silicon and Raspberry Pi users.

Deneb

Core

• Use ROForkchoice in blob verifier (#13426)
• Add Goerli Deneb Fork Epoch (#13390)
• Use deneb key for deneb state in saveStatesEfficientInternal (#13374)
• Initialize Inactivity Scores Correctly (#13375)
• Excluse DA wait time for chain processing time (#13335)
• Initialize sig cache for verification.Initializer (#13295)
• Verify roblobs (#13245)
• KZG Commitment inclusion proof verifier (#13174)
• Merkle Proofs of KZG commitments (#13159)
• Add RO blob sidecar (#13144)
• Check blob index duplication for blob notifier (#13123)
• Remove sidecars with invalid proofs (#13070)
• Proposer: better handling of blobs bundle (#12956)
• Update proposer RPC to new blob sidecar format (#13189)
• Implement Slot-Dependent Caching for Blobs Bundle (#13205)
• Verified roblobs (#13190)

Networking

• Check sidecar index in BlobSidecarsByRoot response (#13180)
• Use proposer index cache for blob verification (#13423)
• VerifiedROBlobs in initial-sync (#13351)
• Reordered blob validation (#13347)
• Initialize blob storage for initial sync service (#13312)
• Use verified blob for gossip checks (#13294)
• Update broadcast method to use BlobSidecar instead of SingedBlobSidecar (#13221)
• Remove pending blobs queue (#13166)
• Reject Blob Sidecar Incorrect Index (#13094)
• Check return and request lengths for blob sidecar by root (#13106)
• Fix blob sidecar subnet check (#13102)
• Add pending blobs queue for missing parent block (#13005)
• Verify blobs that arrived from by root request (#13044)
• Reject blobs with invalid parent (#13047)
• Add more blob and block checks for by range (#13043)
• Exit early if blob by root request is empty (#13038)
• Request missing blobs while processing pending queue (#13015)
• Check blob exists before requesting from peer (#13012)
• Passing block as arugment for sidecar validation (#13062)

Blob Management

• Remove old blob types (#13438)
• minimize syscalls in pruning routine (#13425)
• Prune dangling blob (#13424)
• Use Afero Walk for Pruning Blob (#13410)
• Initialize blob storage without pruning (#13412)
• Fix batch pruning errors (#13355)
• Blob filesystem add pruning during blob write (#13275)
• Blob filesystem add pruning at startup (#13253)
• Ensure partial blob is deleted if there's an error (#13292)
• Split blob pruning into two funcs (#13285)
• Use functional options for --blob-retention-epochs (#13283)
• Blob filesystem: delete blobs (#13233)
• Fix Blob Storage Path (#13222)
• Add blob getters (#13170)
• Blob filesystem: Save Blobs (#13129)
• Blob filesystem: prune blobs (#13147)
• blobstorage: Improve mkdirall error (#13271)

Beacon-API

• Add rpc trigger for blob sidecar event (#13411)
• Do not skip mev boost in v3 block production endpoint (#13365)
• Beacon APIs: re enabling blob events (#13315)
• Beacon API: update Deneb endpoints after removing blob signing (#13235)
• Beacon API: fix get blob returns 500 instead of empty (#13297)
• Fix bug in Beacon API getBlobs (#13100)
• Fix blob_sidecar SSE payload (#13050)
• fix(beacon-chain/rpc): blob_sidecar event stream handler (#12999)
• Improvements to produceBlockV3 (#13027)
• Deneb: Produce Block V3 - adding consensus block value (#12948)

Validator Client

• Validator client: remove blob signing (#13169)
• Deneb - web3signer (#12767)

Testing

• Enable Deneb For E2E Scenario Tests (#13317)
• Activate deneb in E2E (#13311)
• Deneb E2E (#13040)

Miscellaneous

• Update blob pruning log (#13417)
• Fix total pruned metric + add to logging (#13367)
• Check kzg commitment count from builder (#13394)
• Add error wrapping to blob initialization errors (#13366)
• Blob filesystem metrics (#13316)
• Check builder header kzg commitment (#13358)
• Add more color to sending blob by range req log (#13349)
• Move pruning log to after retention check (#13348)
• Enhance Pruning Logs (#13331)
• Rename Blob retention epoch flag (#13124)
• Check that blobs count is correct when unblinding (#13118)
• Log blob's kzg commmitment at sync (#13111)
• Replace MAX_BLOB_EPOCHS usages with more accurate terms (#13098)
• Fix comment of BlobSidecarsBySlot (#13019)

Core Prysm Work(Non-Deneb)

Core Protocol

• Only process blocks which haven't been processed (#13442)
• Initialize exec payload fields and enforce order (#13372)
• Add nil check for head in IsOptimistic (#13439)
• Unlock forkchoice store if attribute is empty (#13427)
• Make Aggregating In Parallel The Permanent Default (#13407)
• Break out several helpers from postBlockProcess (#13419)
• Don't hardcode 4 seconds in forkchoice (#13416)
• Simplify fcu 4 (#13403)
• Remove the getPayloadAttribute call from updateForkchoiceWithExecution (#13402)
• Simplify fcu 2 (#13400)
• Remove getPayloadAttributes from FCU call (#13399)
• Simplify fcu 1 (#13387)
• Remove unsafe proposer indices cache (#13385)
• Rewrite ProposeBlock endpoint (#13380)
• Remove blind field from block type (#13389)
• update shuffling caches before calling FCU on epoch boundaries (#13383)
• Return SignedBeaconBlock from ReadOnlySignedBeaconBlock.Copy (#13386)
• Use advanced epoch cache when preparing proposals (#13377)
• refactor Payload Id caches (#12987)
• Use block value correctly when proposing a block (#13368)
• use different keys for the proposer indices cache (#13272)
• Use a cache of one entry to build attestation (#13300)
• Remove signed block requirement from no-verify functions (#13314)
• Allow requests for old target roots (#13281)
• Remove Redundant Hash Computation in Cache (#13261)
• Fix FFG LMD Consistency Check (Option 2) (#13258)
• Verify lmd without ancestor (#13250)
• Track target in forkchoice (#13249)
• Return early from ReceiveBlock if already sycned (#13089)

Builder

• Adding builder boost factor to get block v3 (#13409)
• Builder API: Fix max field check on toProto function (#13334)
• Add sanity checks for bundle from builder (#13319)
• Update Prysm Proposer end points for Builder API (#13240)
• Builder API: remove blinded blob sidecar (#13202)
• Allow validators registration batching on Builder API /eth/v1/builder/validators (#13178)

State-Management

• Add Detailed Multi Value Metrics (#13429)
• Optimize Multivalue Slice For Trie Recomputation (#13238)
• Fix Multivalue Slice Deadlock (#13087)
• Set Better Slice Capacities in the State (#13068)

Networking

• Refactor Network Config Into Main Config (#13364)
• Handle potential error from newBlockRangeBatcher (#13344)
• Clean Up Goodbye Stream Errors (#13325)
• Support New Subnet Backbone (#13179)
• Increase Networking Defaults (#13278)
• Bump Up Gossip Queue Size (#13277)
• Improve Gossipsub Rejection Metric (#13236)
• Add Gossipsub Queue Flag (#13237)
• Fix Deadlock With Subscriber Checker (#13234)
• Add Additional Pubsub Metrics (#13226)
• Verify Block Signatures On Insertion Into Pending Queue (#13183)
• Enhance Validation for Block by Root RPC Requests (#13184)
• Add a helper for max request block (#13173)
• Fix Pending Queue Deadline Bug (#13145)
• Add context deadline for pending queue's receive block (#13114)
• Fix Pending Queue Expiration Bug (#13104)
• sync only up to previous epoch on phase 1 (#13083)
• Use correct context for sendBatchRootRequest (#13061)
• Refactor Pending Block Queue Logic in Sync Package (#13026)
• Check block exists in pending queue before requesting from peer (#13013)
• Set Verbosity of Goodbye Logs to Trace (#13077)
• use read only head state (#13014)

Beacon-API

Most of the PRs here involve shifting our http endpoints to using vanilla http handlers(without the API middleware).

• http endpoint cleanup (#13432)
• Revert "REST VC: Subscribe to Beacon API events (#13354)" (#13428)
• proposer and attester slashing sse (#13414)
• REST VC: Subscribe to Beacon API events (#13354)
• Simplify error handling for JsonRestHandler (#13369)
• Update block publishing to 2.4.2 spec (#13376)
• Use SkipMevBoost properly during block production (#13352)
• Handle HTTP 404 Not Found in SubmitAggregateAndProof (#13320)
• beacon-chain/rpc: use BalanceAtIndex instead of Balances to reduce memory copy (#13279)
• HTTP endpoints cleanup (#13251)
• APIs: reusing grpc cors middleware for rest (#13284)
• Beacon API: routes unit test (#13276)
• Remove API Middleware (#13243)
• HTTP validator API: beacon and account endpoints (#13191)
• REST VC: Use POST to fetch validators (#13239)
• HTTP handler for Beacon API events (#13207)
• Move weak subjectivity endpoint to HTTP (#13220)
• Handle non-JSON responses from Beacon API (#13213)
• POST version of GetValidators and GetValidatorBalances (#13199)
• [2/5] light client http api (#12984)
• HTTP validator API: wallet endpoints (#13171)
• HTTP Validator API: slashing protection import and export (#13165)
• Config HTTP endpoints (#13168)
• Return 404 from eth/v1/beacon/headers when there are no blocks (#13185)
• Pool slashings HTTP endpoints (#13148)
• Validator HTTP endpoints (#13167)
• Debug HTTP endpoints (#13164)
• HTTP validator API: health endpoints (#13149)
• HTTP Validator API: /eth/v1/keystores (#13113)
• Allow unknown fields in Beacon API responses (#13131)
• HTTP state endpoints (#13099)
• HTTP Validator API: /eth/v1/validator/{pubkey}/feerecipient (#13085)
• HTTP Validator API: /eth/v1/validator/{pubkey}/gas_limit (#13082)
• HTTP VALIDATOR API: remote keymanager api /eth/v1/remotekeys (#13059)
• rpc/apimiddleware: Test all paths can be created (#13073)
• HTTP Beacon APIs for blocks (#13048)
• HTTP VALIDATOR API: /eth/v1/validator/{pubkey}/voluntary_exit (#13032)
• HTTP Beacon APIs: 3 state endpoints (#13001)
• HTTP Beacon APIs for node (#13010)
• HTTP API: /eth/v1/beacon/pool/bls_to_execution_changes (#12963)
• Register sync subnet when fetching sync committee duties through Beacon API (#12972)

Validator Client

• Refactor validator client help. (#13401)
• --validatorS-registration-batch-size (add s) (#13396)
• Validator client: Always use the --datadir value. (#13392)
• Hook to slot stream instead of block stream on the VC (#13327)
• CLI: fixing account import ux bugs (#13328)
• filterAndCacheActiveKeys: Stop filtering out exiting validators (#13305)
• Gracefully handle unknown validator index in the REST VC (#13296)
• Don't fetch duties for unknown keys (#13269)
• Fix Domain Data Caching (#13263)
• Add --jwt-id flag (#13218)
• Make Prysm VC compatible with the version v5.3.0 of the slashing protections interchange tests. (#13232)
• Fix handling POST requests in the REST VC (#13215)
• Better error handling in REST VC (#13203)
• Fix block proposals in the REST validator client (#13116)
• CLEANUP: validator exit prompt (#13057)
• integrate validator count endpoint in validator client (#12912)

Build/CI Work

• Bazel 7.0.0 (#13321)
• Sort static analyzers, add more, fix violations (#13441)
• For golangci-lint, enable all by default (#13353)
• Enable mirror linter and fix findings (#13342)
• Enable usestdlibvars linter and fix findings (#13339)
• Fix docker image version strings in CI (#13356)
• fixing sa4006 (#13350)
• Enable errname linter and fix findings (#13341)
• Remove rules_docker, make multiarch images canonical (#13324)
• Fix staticcheck violations (#13301)
• Add staticchecks to bazel builds (#13298)
• CI: Add merge queue events trigger for github workflows (#13282)
• Update bazel and other CI improvements (#13246)
• bazel: Run buildifier, general cleanup (#13193)
• pgo: Enable pgo behind release flag (#13158)
• pgo: remove default pprof profile (#13150)
• zig: Update zig to recent main branch commit (#13142)
• Enable profile guided optimization for beacon-chain (#13035)
• Refactor Exported Names to Follow Golang Best Practices (#13075)
• Update rules_go and gazelle to 0.42 & 0.33 (latest releases) (#13021)
• Fix image deps (#13022)

Dependency Updates

• Update go to 1.21.6 (#13440)
• Update Our Golang Crypto Library (#13415)
• Update libp2p/go-libp2p-asn-util to v0.4.1 (#13370)
• Update Libp2p To v0.32.1 and Go to v1.21.5 (#13304)
• Bump google.golang.org/grpc from 1.53.0 to 1.56.3 (#13119)
• Update go to 1.20.10 (#13120)

Testing

• Enable Profiling for Long Running E2E Runs (#13421)
• Fetch Goroutine Traces in E2E (#13404)
• Fix Up Builder Evaluator (#13395)
• Increase Blob Batch Parameters in E2E (#13398)
• Uncomment e2e flakiness (#13326)
• Update spectests to 1.4.0-beta.5 (#13318)
• Test improvement TestValidateVoluntaryExit_ValidExit (#13313)
• Simplify post-evaluation in Beacon API evaluator (#13309)
• Run Evaluator In the Middle Of An Epoch (#13303)
• Simplify Beacon API evaluator (#13265)
• Fix Optimistic Sync Evaluator (#13262)
• Add test helpers to produce commitments and proofs (#13242)
• Redesign of Beacon API evaluator (#13229)
• Drop Transaction Count for Transaction Generator (#13228)
• Add concurrency test for getting attestation state (#13196)
• Add construct_generic_block_test to build file (#13195)
• Implement Merkle proof spectests (#13146)
• Remove /node/peers/{peer_id} from Beacon API evaluator (#13138)
• Update spectest and changed minimal preset for field elements (#13090)
• Better Beacon API evaluator part 1 (#13084)
• beacon-chain/blockchain: fix some datarace in go test (#13036)
• beacon-node/rpc: fix go test datarace (#13018)
• Fix Builder Testing For Multiclient Runs (#13091)
• Fill state attestations (#13121)
• beacon-chain/sync: fix some datarace in go test (#13039)
• beacon-chain/execution: fix a data race in testcase (#13016)
• Add state not found test case (#13034)

Feature Updates

• Make New Engine Methods The Permanent Default (#13406)
• Make Reorging Of Late Blocks The Permanent Default (#13405)

Miscellaneous

• Update teku's bootnode (#13437)
• fix metric for exited validator (#13379)
• Fix typos (#13435)
• Replace validator count with validator indices in update fee recipient log (#13384)
• Log value of local payload when proposing (#13381)
• Small encoding fixes on logs and http error code change (#13345)
• typo fix (#13357)
• Fix error string generation for missing commitments (#13338)
• Increase buffer of events channel (#13329)
• Fix missing testnet versions. Issue #13288 (#13323)
• Update README.md (#13302)
• Only run metrics for canonical blocks (#13289)
• Relax file permissions check on existing directories (#13274)
• forkchoice.Getter wrapper with locking wrappers (#13244)
• Initialize cancellable root context in main.go (#13252)
• Fix forkchoice pkg's comments grammar (#13217)
• lock RecentBlockSlot (#13212)
• Comment typo (#13209)
• Optimize ReplayBlocks for Zero Diff (#13198)
• Remove default value of circuit breaker flags (#13186)
• Fix Withdrawals (#13181)
• Remove no-op cancel func (#13069)
• Update Terms of Service (#13163)
• fix head slot in log (#13139)
• DEPRECTATION: Remove exchange transition configuration call (#13127)
• fix segmentation fork when Capella for epoch is MaxUint64 (#13126)
• Return Error Gracefully When Removing 4881 Flag (#13096)
• Add zero length check on indices during NextSyncCommitteeIndices (#13117)
• Replace Empty Slice Literals with Nil Slices (#13093)
• Refactor Error String Formatting According to Go Best Practices (#13092)
• Fix redundant type converstion (#13076)
• docs: fix typo (#13023)
• Add Clarification To Sync Committee Cache (#13067)
• Fix typos (#13053)
• remove bad comment (#13056)
• Remove confusing comment (#13045)
• Log when sending FCU with payload attributes (#13137)
• Fix Withdrawals Marshalling (#13066)
• beacon-chain/execution: no need to reread and unmarshal the eth1Data twice (#12826)