Description:
Polling blockchain RPC nodes individually for real-time liquidity changes introduces high latency and RPC throttling. We need to implement an event-driven ingestion pipeline that streams live block transactions over gRPC into a Kafka broker cluster for async parsing.
Acceptance Criteria:
- Build a gRPC subscriber client connecting to blockchain indexers (e.g., Yellowstone Geyser for Solana or EVM WebSocket streams).
- Produce normalized block events into a partitioned Apache Kafka topic.
- Implement a multi-threaded Rust consumer group that processes transactions asynchronously without blocking the REST API runtime.
Description:
Polling blockchain RPC nodes individually for real-time liquidity changes introduces high latency and RPC throttling. We need to implement an event-driven ingestion pipeline that streams live block transactions over gRPC into a Kafka broker cluster for async parsing.
Acceptance Criteria: