Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add metric for command processor latency #1501

Open
jeffreyscarpenter opened this issue Oct 7, 2024 · 0 comments
Open

Add metric for command processor latency #1501

jeffreyscarpenter opened this issue Oct 7, 2024 · 0 comments

Comments

@jeffreyscarpenter
Copy link
Contributor

In PR #1029 we removed the histograms from the command_processor_process metrics because of the extremely high cardinality that was causing problems for Grafana dashboards and metrics forwarding.

The high cardinality was caused by the multiplicity of tags we supported including command, tenant, sort type, error, error class, error_code, and vector_enabled. When adding histogram buckets to this, it potentially results in a huge number of series.

However it would still be useful to be able to track latency by command for debugging purposes. We should add a new metric called command_processor_latency that is a histogram metric tagged by command and tenant only.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant