-
Notifications
You must be signed in to change notification settings - Fork 1k
feat(blob|state): Blob submission metrics #4664
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
66 commits
Select commit
Hold shift + click to select a range
3090cbf
add blob submission metrics
gupadhyaya b25c77f
fix tests
gupadhyaya c451259
fix build errors
gupadhyaya 931387a
fix api
gupadhyaya 43e6746
shorten
gupadhyaya b4f3dde
Merge branch 'main' into blob_submission_metrics
gupadhyaya 0487333
remove duplicates
gupadhyaya 6be3ead
fix format issue
gupadhyaya 17a30e2
Fix bitswap compilation issues and enable metrics in tastora framework
gupadhyaya 6bc2316
Remove HashOnRead method from BlockstoreWithMetrics
gupadhyaya 962f115
Update tastora framework to use local Docker image with metrics
gupadhyaya 7198aa5
Remove interface compliance check for BlockstoreWithMetrics
gupadhyaya 80e2a6a
fix coreaccessor with new metrics instead of nil, fx.invoke issue
gupadhyaya af3953a
fix blob metrics
gupadhyaya 2af16ef
go.mod changes
gupadhyaya 06e3513
Merge branch 'main' into blob_submission_metrics
gupadhyaya 4659fbe
go mod tidy
gupadhyaya 871bd90
blob metrics fixes
gupadhyaya ccbb3a3
fix
gupadhyaya e00d3c7
remove unnecessary method
gupadhyaya e98716f
remove debug stuff
gupadhyaya ab8d1b2
WithMetrics
gupadhyaya 0f2a3cf
remove passing nil metrics
gupadhyaya 5c1a2f6
private metrics, stop method
gupadhyaya 88ba0e0
cleanup
gupadhyaya 88e78ba
Merge branch 'main' into blob_submission_metrics
gupadhyaya 510b48c
blob metrics changes
gupadhyaya 0044277
remove nil checks
gupadhyaya ab99226
get rid of duplicate legacy state metrics
gupadhyaya 29c9a4e
optimizing metrics recording
gupadhyaya 1ba9c6c
error out if metrics fail
gupadhyaya c604d7b
named error
gupadhyaya 74a42d1
skip resp name
gupadhyaya 878d1bf
error as bool/enum to avoid cardinality explosion
gupadhyaya 9356b77
golangci-lint fix
gupadhyaya 06a3961
unify error counting
gupadhyaya b6f7a3d
remove magic strings
gupadhyaya 96b085a
golangci-lint fixes
gupadhyaya f110c2d
further fix golangci-lint
gupadhyaya 533f70e
fixing docker security issue
gupadhyaya 344da58
Revert "fixing docker security issue"
gupadhyaya f7d2d64
Merge origin/main into blob_submission_metrics
gupadhyaya 0de2e67
Fix compilation error after merge
gupadhyaya 380ea9e
Apply goimports-reviser formatting to fix import organization
gupadhyaya 5ac90dc
Fix import ordering in state/core_access.go
gupadhyaya b81bfc0
Fix remaining issues after merge
gupadhyaya 4495e04
minor
gupadhyaya 5006f04
go mod tidy
gupadhyaya 0d3d4e4
remove unnecessary metrics
gupadhyaya 70ad967
Merge branch 'main' into blob_submission_metrics
gupadhyaya c373bd9
Remove accidentally committed grafana and metrics files
gupadhyaya 5ac205d
Fix TestListenerWithWrongChainRPC cleanup issue
gupadhyaya 03d26e7
Merge remote-tracking branch 'origin/main' into blob_submission_metrics
gupadhyaya ace5b72
fix
gupadhyaya 24a6adc
minor fix
gupadhyaya a0079b2
fix failing test
gupadhyaya 592d504
cleanup
gupadhyaya 086d7c7
remove redundant metrics, private methods, cleanup
gupadhyaya ac2a6f6
Merge branch 'main' into blob_submission_metrics
gupadhyaya 3b73c24
trash clean
gupadhyaya d111c17
use normal counter, fix cardinality issue
gupadhyaya e0954dd
Merge branch 'main' into blob_submission_metrics
gupadhyaya 54cb5cf
add gas tracking for all txs
gupadhyaya 11cc8cb
Merge branch 'main' into blob_submission_metrics
gupadhyaya 7a2515f
fix lint
gupadhyaya 153e9a2
fix race
gupadhyaya File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,128 @@ | ||
| package blob | ||
|
|
||
| import ( | ||
| "context" | ||
| "errors" | ||
| "time" | ||
|
|
||
| "go.opentelemetry.io/otel" | ||
| "go.opentelemetry.io/otel/attribute" | ||
| "go.opentelemetry.io/otel/metric" | ||
| ) | ||
|
|
||
| var meter = otel.Meter("blob") | ||
|
|
||
| // Error type constants for metrics labels | ||
| const ( | ||
| errorTypeNotFound = "not_found" | ||
| errorTypeTimeout = "timeout" | ||
| errorTypeCanceled = "canceled" | ||
| errorTypeUnknown = "unknown" | ||
| ) | ||
|
|
||
| // metrics tracks blob-related metrics | ||
| type metrics struct { | ||
| // Retrieval metrics | ||
| retrievalDuration metric.Float64Histogram | ||
| retrievalTotal metric.Int64Counter | ||
|
|
||
| // Proof metrics | ||
| proofDuration metric.Float64Histogram | ||
gupadhyaya marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| proofTotal metric.Int64Counter | ||
| } | ||
|
|
||
| // WithMetrics initializes metrics for the Service | ||
| func (s *Service) WithMetrics() error { | ||
| // Retrieval metrics | ||
| retrievalDuration, err := meter.Float64Histogram( | ||
| "blob_retrieval_duration_seconds", | ||
| metric.WithDescription("Duration of blob retrieval operations"), | ||
| metric.WithUnit("s"), | ||
| ) | ||
| if err != nil { | ||
| return err | ||
| } | ||
|
|
||
| // Proof metrics | ||
| proofDuration, err := meter.Float64Histogram( | ||
| "blob_proof_duration_seconds", | ||
| metric.WithDescription("Duration of blob proof operations"), | ||
| metric.WithUnit("s"), | ||
| ) | ||
| if err != nil { | ||
| return err | ||
| } | ||
|
|
||
| retrievalTotal, err := meter.Int64Counter( | ||
| "blob_retrieval_total", | ||
| metric.WithDescription("Total number of blob retrievals"), | ||
| ) | ||
| if err != nil { | ||
| return err | ||
| } | ||
|
|
||
| proofTotal, err := meter.Int64Counter( | ||
| "blob_proof_total", | ||
| metric.WithDescription("Total number of blob proofs"), | ||
| ) | ||
| if err != nil { | ||
| return err | ||
| } | ||
|
|
||
| m := &metrics{ | ||
| retrievalDuration: retrievalDuration, | ||
| retrievalTotal: retrievalTotal, | ||
| proofDuration: proofDuration, | ||
| proofTotal: proofTotal, | ||
| } | ||
|
|
||
| s.metrics = m | ||
| return nil | ||
| } | ||
|
|
||
| // observeRetrieval records blob retrieval metrics | ||
| func (m *metrics) observeRetrieval(ctx context.Context, duration time.Duration, err error) { | ||
| if m == nil { | ||
| return | ||
| } | ||
|
|
||
| // Record metrics with error type enum to avoid cardinality explosion | ||
| attrs := []attribute.KeyValue{} | ||
| if err != nil { | ||
| errorType := errorTypeUnknown | ||
| switch { | ||
| case errors.Is(err, ErrBlobNotFound): | ||
| errorType = errorTypeNotFound | ||
| case errors.Is(err, context.DeadlineExceeded): | ||
| errorType = errorTypeTimeout | ||
| case errors.Is(err, context.Canceled): | ||
| errorType = errorTypeCanceled | ||
| } | ||
| attrs = append(attrs, attribute.String("error_type", errorType)) | ||
| } | ||
|
|
||
| m.retrievalDuration.Record(ctx, duration.Seconds(), metric.WithAttributes(attrs...)) | ||
| m.retrievalTotal.Add(ctx, 1, metric.WithAttributes(attrs...)) | ||
| } | ||
|
|
||
| // observeProof records blob proof metrics | ||
| func (m *metrics) observeProof(ctx context.Context, duration time.Duration, err error) { | ||
| if m == nil { | ||
| return | ||
| } | ||
|
|
||
| attrs := []attribute.KeyValue{} | ||
| if err != nil { | ||
| errorType := errorTypeUnknown | ||
| switch { | ||
| case errors.Is(err, context.DeadlineExceeded): | ||
| errorType = errorTypeTimeout | ||
| case errors.Is(err, context.Canceled): | ||
| errorType = errorTypeCanceled | ||
| } | ||
| attrs = append(attrs, attribute.String("error_type", errorType)) | ||
| } | ||
|
|
||
| m.proofDuration.Record(ctx, duration.Seconds(), metric.WithAttributes(attrs...)) | ||
| m.proofTotal.Add(ctx, 1, metric.WithAttributes(attrs...)) | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.