-
Notifications
You must be signed in to change notification settings - Fork 120
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Block submission metrics #625
Conversation
9781986
to
4ab0fa0
Compare
services/api/service.go
Outdated
otelapi.WithAttributes( | ||
attribute.Int64("slot", s), | ||
attribute.Int64("floorSecIntoSlot", msIntoSlot/1000), | ||
), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
remark:
not so relevant if we just drop this counter, but generally this approach is not desirable as each new combination of labels (slot
and floorSecIntoSlot
in this case) will cause prometheus to start a fresh counter (which will eventually explode as the size of the scrape payload will keep growing with time).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
how do we label metrics not using attributes then?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
there will be labels added automatically by the scraper (things like instance
, instance_name
, and so on).
important is to keep all possible label permutations limited.
e.g. in the current case floorSecIntoSlot
would be still fine (and maybe interesting to capture, to be able to see the the breakdown of requests per second => and there would only be 12 possible labels for that.
but the slot
would change every 12s => therefore infinite count of labels. unless we somehow make the scraper "forget" old labels, so that scrape request size would be limited in size.
9e59e24
to
e95e2fe
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
let's goooo
📝 Summary
Migrating the metrics to OpenTelemetry and set up more detailed profiling to measure the request latencies of block submission and proposer api.
⛱ Motivation and Context
Improve observability and latency issues in boost relay
📚 References
✅ I have run these commands
make lint
make test-race
go mod tidy
CONTRIBUTING.md