feat: p2p exchange wrapper

[tac0turtle]

Overview

This pr was an idea from @renaynay in which we create an exchange wrapper so that the exchange can check our store before making public requests for data. This would help the edge case i ran into today where the p2p store was missing 40 blocks randomly

[github-actions]

The latest Buf updates on your PR. Results from workflow CI / buf-check (pull_request).

Build	Format	Lint	Breaking	Updated (UTC)
✅ passed	⏩ skipped	✅ passed	✅ passed	Nov 25, 2025, 2:28 PM

[claude]

Claude finished @tac0turtle's task —— View job

---

Code Review Complete

Summary

This PR introduces an exchange wrapper that checks the local DA store before making P2P network requests for headers and block data. This is a valuable optimization that addresses the issue of missing blocks in the P2P store. The implementation is clean and follows good Go patterns, but there are several areas for improvement.

---

🔴 Critical Issues

1. Performance Issue: Using GetBlockByHash Instead of GetHeader for Headers

Location: pkg/sync/sync_service.go:102-105

getter := func(ctx context.Context, s store.Store, hash header.Hash) (*types.SignedHeader, error) {
    h, _, err := s.GetBlockByHash(ctx, hash)
    return h, err
}

Issue: For the HeaderSyncService, the getter function calls GetBlockByHash which retrieves both the header AND the data, but then discards the data. Looking at the store implementation, GetBlockData calls GetHeader internally and then fetches additional data.

Impact: This unnecessarily reads and deserializes block data that's immediately discarded, wasting I/O and CPU.

Recommendation: Create a GetHeaderByHash method in the store interface, or use type switching to call the appropriate method. As @julienrbrt noted in the review comment, GetHeader is less expensive.

Suggested Fix:

// In pkg/store/types.go, add to Reader interface:
GetHeaderByHash(ctx context.Context, hash []byte) (*types.SignedHeader, error)

// In pkg/sync/sync_service.go:
getter := func(ctx context.Context, s store.Store, hash header.Hash) (*types.SignedHeader, error) {
    return s.GetHeaderByHash(ctx, hash)
}

---

⚠️ Major Issues

2. Inconsistent Error Handling Between Old and New Implementation

Location: pkg/sync/exchange_wrapper.go:20-42

Old Implementation (using type switch):

func (ew *exchangeWrapper[H]) Get(ctx context.Context, hash header.Hash) (H, error) {
    var zero H
    if ew.daStore != nil {
        switch any(zero).(type) {
        case *types.SignedHeader:
            h, _, err := ew.daStore.GetBlockByHash(ctx, hash)
            if err == nil && h != nil {
                return any(h).(H), nil
            }
        // ...
    }
    return ew.Exchange.Get(ctx, hash)
}

New Implementation (using function injection):

func (ew *exchangeWrapper[H]) Get(ctx context.Context, hash header.Hash) (H, error) {
    if ew.daStore != nil && ew.getter != nil {
        if h, err := ew.getter(ctx, ew.daStore, hash); err == nil && !h.IsZero() {
            return h, nil
        }
    }
    return ew.Exchange.Get(ctx, hash)
}

Differences:

1. Old: Checked h != nil
2. New: Checks !h.IsZero()

Concern: While IsZero() is more idiomatic for the header.Header interface, the silent error swallowing (err == nil) means we fall back to P2P for any error, including serious errors like context cancellation, database corruption, or permission issues.

Recommendation: Log store errors at debug/warn level to aid troubleshooting:

if ew.daStore != nil && ew.getter != nil {
    h, err := ew.getter(ctx, ew.daStore, hash)
    if err != nil {
        // Log for observability, but still fall back to P2P
        // Consider logging at warn level for unexpected errors
    }
    if !h.IsZero() {
        return h, nil
    }
}

3. Race Condition Risk: daStore Can Be Nil After Check

Location: pkg/sync/exchange_wrapper.go:22, 34

if ew.daStore != nil && ew.getter != nil {
    if h, err := ew.getter(ctx, ew.daStore, hash); err == nil && !h.IsZero() {

Issue: While unlikely in this codebase, ew.daStore is not protected by any synchronization. If it were set to nil concurrently after the check, ew.getter would receive a nil store.

Recommendation: Given that the wrapper is created once at initialization and not modified, this is acceptable. However, adding a comment would clarify the concurrency assumptions:

// exchangeWrapper wraps a header.Exchange to check local DA store before network requests.
// Fields are set once at initialization and must not be modified after creation.
type exchangeWrapper[H header.Header[H]] struct {

---

💡 Code Quality & Architecture

4. Excellent Refactoring from Type Switching to Function Injection

Location: pkg/sync/exchange_wrapper.go

Positive: The new implementation eliminates runtime type switches in favor of compile-time function injection. This is a significant improvement:

• Better testability: Functions can be mocked/stubbed easily
• Better performance: No runtime type assertions
• Better separation of concerns: Store access logic is defined at service creation
• Type safety: Compile-time checking instead of runtime panics

5. Function Type Aliases Improve Readability

Location: pkg/sync/exchange_wrapper.go:10-11

type storeGetter[H header.Header[H]] func(context.Context, store.Store, header.Hash) (H, error)
type storeGetterByHeight[H header.Header[H]] func(context.Context, store.Store, uint64) (H, error)

Positive: These type aliases make the wrapper's field declarations much more readable. Good practice.

6. Consistent Nil Checks

Location: pkg/sync/exchange_wrapper.go:22, 34

Positive: The code consistently checks both daStore and the getter functions before use, preventing nil pointer panics.

---

🧪 Testing

7. Test Coverage Is Good But Could Be More Comprehensive

Location: pkg/sync/exchange_wrapper_test.go

Strengths:

• Tests cover the happy path (hit in store)
• Tests cover the fallback path (miss in store)
• Tests cover the nil store path

Missing Test Cases:

1. Context cancellation: What happens if the context is canceled during store lookup?
2. Store returns zero value with nil error: Edge case where err == nil but header is zero
3. Concurrent access: While not expected to be an issue, a basic race detector test would be valuable
4. GetByHeight store error: Only tests Get with store errors, not GetByHeight

Recommendation: Add these test cases:

t.Run("Context Canceled During Store Lookup", func(t *testing.T) {
    ctx, cancel := context.WithCancel(context.Background())
    cancel() // Cancel immediately
    
    getter := func(ctx context.Context, s store.Store, h header.Hash) (*types.SignedHeader, error) {
        return nil, ctx.Err()
    }
    
    mockEx := extmocks.NewMockExchange[*types.SignedHeader](t)
    mockEx.On("Get", ctx, hash).Return(expectedHeader, context.Canceled)
    
    ew := &exchangeWrapper[*types.SignedHeader]{
        Exchange: mockEx,
        daStore:  mocks.NewMockStore(t),
        getter:   getter,
    }
    
    h, err := ew.Get(ctx, hash)
    assert.Error(t, err)
    assert.True(t, errors.Is(err, context.Canceled))
})

t.Run("GetByHeight Store Error Falls Back to Exchange", func(t *testing.T) {
    // Similar to "Miss in Store" but for GetByHeight
})

8. Mock Generation Configuration Is Correct

Location: .mockery.yaml:42-48

Positive: The addition of the Exchange interface mock from go-header is properly configured and will enable better testing of the wrapper.

---

🔒 Security

9. No Security Issues Identified

The wrapper doesn't handle credentials, doesn't expose sensitive data, and doesn't introduce new attack surfaces. The fallback to P2P is fail-open, which is appropriate for this use case.

---

⚡ Performance

10. Wrapper Adds Minimal Overhead

Location: pkg/sync/exchange_wrapper.go:20-42

Analysis:

• Added overhead: 2 nil checks + 1 function call + 1 IsZero() check
• This is negligible compared to store I/O or network requests
• Cache hit: Eliminates network roundtrip (huge win)
• Cache miss: Adds ~microseconds of overhead (acceptable)

Overall Assessment: Performance impact is excellent. The cache hits will significantly reduce P2P load.

11. Potential Issue: No Metrics/Observability

Location: pkg/sync/exchange_wrapper.go

Issue: There's no way to observe:

• Cache hit rate
• Cache miss rate
• Store error rate
• Fallback frequency

Recommendation: Add metrics (if not too much complexity for this PR):

// In exchangeWrapper, add metric counters
cacheHits   atomic.Uint64
cacheMisses atomic.Uint64
storeErrors atomic.Uint64

// In Get/GetByHeight
if !h.IsZero() {
    ew.cacheHits.Add(1)
    return h, nil
}
ew.cacheMisses.Add(1)

Or at minimum, add debug logging:

if !h.IsZero() {
    // Consider adding: logger.Debug().Str("hash", hash.String()).Msg("cache hit")
    return h, nil
}

---

📝 Documentation

12. Missing Package-Level and Type-Level Documentation

Location: pkg/sync/exchange_wrapper.go:13-18

Issue: The exchangeWrapper type and its fields lack godoc comments.

Recommendation: Add documentation:

// exchangeWrapper wraps a header.Exchange to check the local DA store before
// making network requests. This optimizes P2P usage by serving requests from
// local storage when available.
//
// The wrapper is created once at initialization. All fields are immutable after
// creation and safe for concurrent access.
type exchangeWrapper[H header.Header[H]] struct {
    header.Exchange[H]
    daStore        store.Store            // Local DA store to check before network requests
    getter         storeGetter[H]         // Function to retrieve headers/data by hash from store
    getterByHeight storeGetterByHeight[H] // Function to retrieve headers/data by height from store
}

13. Function Comments Could Be More Descriptive

Location: pkg/sync/exchange_wrapper.go:20, 32

Current:

func (ew *exchangeWrapper[H]) Get(ctx context.Context, hash header.Hash) (H, error) {
    // Check DA store first

Recommendation:

// Get retrieves a header by hash, checking the local DA store first before
// falling back to the network exchange. Store errors are silently ignored
// and trigger fallback to the network.
func (ew *exchangeWrapper[H]) Get(ctx context.Context, hash header.Hash) (H, error) {

---

🎯 Additional Recommendations

14. Consider Adding Integration Test

Location: pkg/sync/sync_service_test.go

The existing integration tests don't verify the new wrapper behavior in a realistic scenario. Consider adding a test that:

1. Populates the DA store with some blocks
2. Verifies that GetByHeight and Get serve from cache
3. Verifies fallback to P2P for missing blocks

15. Consider Making Wrapper Reusable

Location: pkg/sync/exchange_wrapper.go

Opportunity: The wrapper is currently private (exchangeWrapper). If this pattern proves valuable, consider:

• Making it public: ExchangeWrapper
• Moving it to a shared package
• Adding constructor function for clarity

This could benefit other parts of the codebase that use header.Exchange.

---

📊 Test Coverage Analysis

Codecov Report: 70.83% patch coverage with 14 lines missing in sync_service.go

Lines Missing Coverage (per Codecov):
Looking at sync_service.go:102-109, these are the new getter functions. The missing coverage is likely:

1. Error paths in the getter functions
2. The wrapper construction in setupP2PInfrastructure

Recommendation: The new exchange_wrapper_test.go tests the wrapper thoroughly, which is good. The integration tests should exercise the wired-up paths in sync_service.go.

---

✅ Positive Aspects Worth Highlighting

1. Solves Real Problem: Addresses the production issue of 40 randomly missing blocks
2. Clean Design: Function injection is more elegant than type switching
3. Backward Compatible: Doesn't break existing behavior, pure enhancement
4. Well Tested: New wrapper has dedicated unit tests
5. Minimal Changes: Focused PR that doesn't over-engineer
6. Type Safe: Leverages Go generics effectively

---

🏁 Conclusion

This is a solid PR that solves a real problem with a clean design. The main concerns are:

1. Performance: Use GetHeader instead of GetBlockByHash for header lookups (per @julienrbrt's comment)
2. Observability: Consider adding metrics or debug logging for cache hit/miss
3. Error Handling: Log store errors for troubleshooting
4. Documentation: Add godoc comments

Recommendation: Approve with minor changes requested (primarily #1 and #4)

---

[codecov]

Codecov Report

❌ Patch coverage is 70.83333% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 64.81%. Comparing base (830ae45) to head (168c8cb).

Files with missing lines	Patch %	Lines
pkg/sync/sync_service.go	56.25%	10 Missing and 4 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2855      +/-   ##
==========================================
+ Coverage   64.76%   64.81%   +0.05%     
==========================================
  Files          81       82       +1     
  Lines        7350     7379      +29     
==========================================
+ Hits         4760     4783      +23     
- Misses       2049     2052       +3     
- Partials      541      544       +3     

Flag	Coverage Δ
combined	64.81% <70.83%> (+0.05%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.