fix(spo): apply review comments and coding style to ProtoFire SPO code by cosmir17 · Pull Request #661 · midnightntwrk/midnight-indexer

cosmir17 · 2026-01-13T13:24:46Z

Summary

Apply review comments from PR feat: integrate spo indexer #526 (SQL file naming, SIGTERM handling, Blockfrost ID security)
Align with indexer coding style patterns
Add GraphQL schema generation CLI for spo-api

Context

This PR shows changes made on top of ProtoFire's SPO indexer code.
Base branch contains main branch codes on top of the ProtoFire's original code.

Changes

fix: update reqwest feature for 0.13 compatibility
refactor: apply PR feat: integrate spo indexer #526 review comments
style: apply indexer code style patterns
feat: add spo-api schema generation CLI

Test

CI tests pass.
Tests deferred per discussion with Heiko (insource first, tests later) due to the team priority.

…3 compatibility

- Replace println!/eprintln! with log::{debug, error, info, warn} - Replace .unwrap() with .expect() with meaningful messages - Add SIGTERM handling to spo-indexer (graceful shutdown) - Fix telemetry service_name: "chain-indexer" → "spo-indexer" - Remove Blockfrost ID from config.yaml (use env var instead) - Consolidate SPO SQL migrations into 003_spo.sql - Remove unused imports in spo-indexer

Apply coding conventions: lowercase logs, inline format args, turbofish, to_owned(), doc comment periods, and imports at file top.

Add spo-api-cli binary with print-api-schema-v1 command and justfile target to generate spo-api/graphql/schema-v1.graphql.

hseeberger · 2026-01-14T13:50:21Z

Can we skip the extra API and move the code from spo-api into indexer-api?

spo-api/src/application.rs

indexer-common/migrations/postgres/003_spo.sql

spo-indexer/src/domain/storage.rs

spo-indexer/src/infra/storage.rs

spo-indexer/src/utils.rs

spo-indexer/src/infra/subxt_node.rs

spo-indexer/src/application.rs

spo-indexer/config.yaml

Apply coding style fixes including SIGTERM casing, typed APIs, proper error handling, and import patterns. Consolidate SPO migrations into 001_initial.sql.

cosmir17 · 2026-01-19T14:40:31Z

Can we skip the extra API and move the code from spo-api into indexer-api?

~~I have made a branch and changes for your request. The PR is #674.~~
~~I am merging into this PR.~~
Thanks for your review. I have merged.

* refactor(spo): merge spo-api into indexer-api Move SPO GraphQL queries from standalone spo-api crate into indexer-api, following the existing storage trait abstraction pattern. This consolidates the API surface and eliminates the need for a separate SPO API service. Changes: - Add SPO domain types (SpoIdentity, PoolMetadata, EpochPerf, etc.) - Add SpoStorage trait with 18 storage methods - Implement SpoStorage for PostgreSQL (cloud) and SQLite (standalone) - Add SpoQuery GraphQL resolver with 20 query methods - Use MergedObject to combine Query and SpoQuery - Remove spo-api from workspace members - Update GraphQL schema with SPO queries * refactor(spo): fold SpoQuery into Query Move all SPO query methods from separate SpoQuery struct into the main Query struct, eliminating the need for MergedObject. This simplifies the GraphQL schema structure as suggested in PR review.

cosmir17 · 2026-01-20T18:34:58Z

Can we skip the extra API and move the code from spo-api into indexer-api?

Done. Merged spo-api into indexer-api and folded all SPO queries into Query.

indexer-common/migrations/postgres/001_initial.sql

hseeberger · 2026-01-23T07:36:02Z

indexer-common/migrations/postgres/001_initial.sql

+    UNIQUE (spo_sk, epoch_no)
+);
+
+-- Update "updated_at" field each time the record is updated


Uh! I don't like "code" in the database. Would our other databases support the same? Could we move setting the updated_at to the application logic? Maybe not for this PR, but let's at least create a ticket to improve this later.

This trigger was in the original ProtoFire implementation. Agreed it's not ideal. I'll create a ticket to move updated_at logic to application layer. SQLite doesn't support triggers the same way, so this would need rework anyway for standalone mode.

I have created a ticket: https://shielded.atlassian.net/browse/PM-21550

indexer-common/migrations/postgres/001_initial.sql

spo-api/src/infra/api/mod.rs

cosmir17 · 2026-01-23T13:10:27Z

I noticed the SPO tables are only in the PostgreSQL migration - the original ProtoFire code didn't include SQLite migrations either. However, the SPO queries are now in indexer-api which is used by both cloud and standalone modes.

Should we:

Add SPO tables to SQLite (queries would work but return empty - no spo-indexer in standalone)
Leave as-is (SPO is cloud-only, standalone would get SQL errors if SPO queries are called)
Feature-gate the SPO queries to exclude them from standalone builds
?

which option do you think most feasible?

hseeberger · 2026-01-24T17:05:12Z

which option do you think most feasible?

Whichever is the least effort for now and makes sure we still can build standalone. WDYT?

hseeberger

I believe quality now is good enough; but we need to further work on the codebase in the future.

hseeberger

We cannot merge this yet, because of feature freeze for the next release.

hseeberger · 2026-01-26T10:31:45Z

Converted to draft, because we cannot merge it yet.

cosmir17 · 2026-01-26T18:03:04Z

which option do you think most feasible?

Whichever is the least effort for now and makes sure we still can build standalone. WDYT?

I think adding SPO tables to the SQLite migration is the least effort. The tables will be empty in standalone mode (no spo-indexer), but queries will work without SQL errors. What do you think?

- Delete spo-api/ directory (merged into indexer-api in PR #674) - Delete outdated MIGRATION.md (ProtoFire migration docs) - Remove spo-api service from docker-compose.yaml - Add SPO tables to SQLite migration for standalone mode

#661) * fix: update reqwest feature from rustls-tls to rustls for reqwest 0.13 compatibility * refactor(spo): apply PR #526 review comments - Replace println!/eprintln! with log::{debug, error, info, warn} - Replace .unwrap() with .expect() with meaningful messages - Add SIGTERM handling to spo-indexer (graceful shutdown) - Fix telemetry service_name: "chain-indexer" → "spo-indexer" - Remove Blockfrost ID from config.yaml (use env var instead) - Consolidate SPO SQL migrations into 003_spo.sql - Remove unused imports in spo-indexer * style(spo): apply indexer code style patterns Apply coding conventions: lowercase logs, inline format args, turbofish, to_owned(), doc comment periods, and imports at file top. * feat(spo-api): add GraphQL schema generation CLI Add spo-api-cli binary with print-api-schema-v1 command and justfile target to generate spo-api/graphql/schema-v1.graphql. * refactor(spo): address PR #661 review comments Apply coding style fixes including SIGTERM casing, typed APIs, proper error handling, and import patterns. Consolidate SPO migrations into 001_initial.sql. * refactor(spo): merge spo-api into indexer-api (#674) * refactor(spo): merge spo-api into indexer-api Move SPO GraphQL queries from standalone spo-api crate into indexer-api, following the existing storage trait abstraction pattern. This consolidates the API surface and eliminates the need for a separate SPO API service. Changes: - Add SPO domain types (SpoIdentity, PoolMetadata, EpochPerf, etc.) - Add SpoStorage trait with 18 storage methods - Implement SpoStorage for PostgreSQL (cloud) and SQLite (standalone) - Add SpoQuery GraphQL resolver with 20 query methods - Use MergedObject to combine Query and SpoQuery - Remove spo-api from workspace members - Update GraphQL schema with SPO queries * refactor(spo): fold SpoQuery into Query Move all SPO query methods from separate SpoQuery struct into the main Query struct, eliminating the need for MergedObject. This simplifies the GraphQL schema structure as suggested in PR review. * refactor(spo): address PR #661 review comments (2nd round) * refactor(spo): delete spo-api crate and add SQLite SPO tables - Delete spo-api/ directory (merged into indexer-api in PR #674) - Delete outdated MIGRATION.md (ProtoFire migration docs) - Remove spo-api service from docker-compose.yaml - Add SPO tables to SQLite migration for standalone mode

* feat(spo): integrate spo indexer Originally authored by caparrosm (ProtoFire). Commit was unsigned and blocked PR merge due to repo requiring verified signatures. Amended author and re-signed to unblock. * fix(spo): apply review comments and coding style to ProtoFire SPO code (#661) * fix: update reqwest feature from rustls-tls to rustls for reqwest 0.13 compatibility * refactor(spo): apply PR #526 review comments - Replace println!/eprintln! with log::{debug, error, info, warn} - Replace .unwrap() with .expect() with meaningful messages - Add SIGTERM handling to spo-indexer (graceful shutdown) - Fix telemetry service_name: "chain-indexer" → "spo-indexer" - Remove Blockfrost ID from config.yaml (use env var instead) - Consolidate SPO SQL migrations into 003_spo.sql - Remove unused imports in spo-indexer * style(spo): apply indexer code style patterns Apply coding conventions: lowercase logs, inline format args, turbofish, to_owned(), doc comment periods, and imports at file top. * feat(spo-api): add GraphQL schema generation CLI Add spo-api-cli binary with print-api-schema-v1 command and justfile target to generate spo-api/graphql/schema-v1.graphql. * refactor(spo): address PR #661 review comments Apply coding style fixes including SIGTERM casing, typed APIs, proper error handling, and import patterns. Consolidate SPO migrations into 001_initial.sql. * refactor(spo): merge spo-api into indexer-api (#674) * refactor(spo): merge spo-api into indexer-api Move SPO GraphQL queries from standalone spo-api crate into indexer-api, following the existing storage trait abstraction pattern. This consolidates the API surface and eliminates the need for a separate SPO API service. Changes: - Add SPO domain types (SpoIdentity, PoolMetadata, EpochPerf, etc.) - Add SpoStorage trait with 18 storage methods - Implement SpoStorage for PostgreSQL (cloud) and SQLite (standalone) - Add SpoQuery GraphQL resolver with 20 query methods - Use MergedObject to combine Query and SpoQuery - Remove spo-api from workspace members - Update GraphQL schema with SPO queries * refactor(spo): fold SpoQuery into Query Move all SPO query methods from separate SpoQuery struct into the main Query struct, eliminating the need for MergedObject. This simplifies the GraphQL schema structure as suggested in PR review. * refactor(spo): address PR #661 review comments (2nd round) * refactor(spo): delete spo-api crate and add SQLite SPO tables - Delete spo-api/ directory (merged into indexer-api in PR #674) - Delete outdated MIGRATION.md (ProtoFire migration docs) - Remove spo-api service from docker-compose.yaml - Add SPO tables to SQLite migration for standalone mode * refactor(spo): move updated_at logic from DB trigger to application layer (PM-21550) (#725) * fix(spo): replace deprecated sidechain_getAriadneParameters RPC (#727) Use systemParameters_getAriadneParameters which sources the D Parameter from the on-chain pallet instead of Cardano. * refactor(spo): remove static metadata and OnlineClient from spo-indexer (#731) * refactor(spo): remove static metadata and OnlineClient from spo-indexer Delete ProtoFire's polkadot_metadata.scale, subxt macro, and build.rs. Replace get_block_timestamp with a DB query on the blocks table (populated by chain-indexer) and rewrite get_epoch_duration to use LegacyRpcMethods instead of OnlineClient. * feat(spo): add run-spo-indexer command to justfile * style(spo): apply rustfmt formatting * refactor: apply review feedback on get_block_timestamp * fix(spo): remove MIN_COMMITTEE_SIZE retry loop from get_committee The infinite retry loop waited for >= 300 committee members before proceeding, causing the spo-indexer to hang on all non-mainnet environments where committee sizes are much smaller (e.g. 6-10). * fix(spo): align Dockerfile with other indexer components * ci: add spo-indexer to Docker image build matrix (#744) * update Cargo.lock

cosmir17 added 4 commits January 9, 2026 19:07

fix: update reqwest feature from rustls-tls to rustls for reqwest 0.1…

6e95265

…3 compatibility

style(spo): apply indexer code style patterns

d2a489f

Apply coding conventions: lowercase logs, inline format args, turbofish, to_owned(), doc comment periods, and imports at file top.

feat(spo-api): add GraphQL schema generation CLI

ddc0cb6

Add spo-api-cli binary with print-api-schema-v1 command and justfile target to generate spo-api/graphql/schema-v1.graphql.

cosmir17 self-assigned this Jan 13, 2026

cosmir17 requested a review from a team as a code owner January 13, 2026 13:24

cosmir17 requested a review from hseeberger January 13, 2026 13:35

hseeberger requested changes Jan 14, 2026

View reviewed changes

refactor(spo): address PR #661 review comments

8989096

Apply coding style fixes including SIGTERM casing, typed APIs, proper error handling, and import patterns. Consolidate SPO migrations into 001_initial.sql.

cosmir17 mentioned this pull request Jan 19, 2026

refactor(spo): merge spo-api into indexer-api #674

Merged

hseeberger requested changes Jan 23, 2026

View reviewed changes

cosmir17 added a commit that referenced this pull request Jan 23, 2026

refactor(spo): address PR #661 review comments (2nd round)

5826855

cosmir17 requested a review from hseeberger January 23, 2026 13:11

refactor(spo): address PR #661 review comments (2nd round)

341d549

cosmir17 force-pushed the feat/insource-spo-indexer branch from 5826855 to 341d549 Compare January 23, 2026 15:55

hseeberger approved these changes Jan 24, 2026

View reviewed changes

hseeberger requested changes Jan 26, 2026

View reviewed changes

hseeberger marked this pull request as draft January 26, 2026 10:31

cosmir17 requested a review from hseeberger January 29, 2026 15:31

hseeberger marked this pull request as ready for review January 29, 2026 15:33

hseeberger approved these changes Jan 29, 2026

View reviewed changes

cosmir17 merged commit b2558f6 into feat/ledger7-protofire-integrate-spo-services-without-sean-changes Jan 29, 2026
13 checks passed

cosmir17 deleted the feat/insource-spo-indexer branch January 29, 2026 15:37

This was referenced Jan 31, 2026

feat(spo): integrate SPO indexer into midnight-indexer #735

Merged

feat: integrate spo indexer #526

Closed

Conversation

cosmir17 commented Jan 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Context

Changes

Test

Uh oh!

hseeberger commented Jan 14, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cosmir17 commented Jan 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cosmir17 commented Jan 20, 2026

Uh oh!

Uh oh!

Uh oh!

hseeberger Jan 23, 2026

Choose a reason for hiding this comment

Uh oh!

cosmir17 Jan 23, 2026

Choose a reason for hiding this comment

Uh oh!

cosmir17 Jan 23, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cosmir17 commented Jan 23, 2026

Uh oh!

hseeberger commented Jan 24, 2026

Uh oh!

hseeberger left a comment

Choose a reason for hiding this comment

Uh oh!

hseeberger left a comment

Choose a reason for hiding this comment

Uh oh!

hseeberger commented Jan 26, 2026

Uh oh!

cosmir17 commented Jan 26, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

cosmir17 commented Jan 13, 2026 •

edited

Loading

cosmir17 commented Jan 19, 2026 •

edited

Loading