Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DocDB] Create kLatest Metric Aggregation Function #25437

Open
1 task done
yusong-yan opened this issue Dec 26, 2024 · 0 comments
Open
1 task done

[DocDB] Create kLatest Metric Aggregation Function #25437

yusong-yan opened this issue Dec 26, 2024 · 0 comments
Labels
area/docdb YugabyteDB core features kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue status/awaiting-triage Issue awaiting triage

Comments

@yusong-yan
Copy link
Contributor

yusong-yan commented Dec 26, 2024

Jira Link: DB-14674

Description

As discussed in #25422, many xCluster metrics use kSum aggregation, which leads to overflows and inaccuracies, particularly for metrics like last_read_hybridtime, As a short-term solution, we are switching to use maximum aggregation kMax .
This issue proposes a longer-term solution: implementing a kLatest aggregation function to provide the most accurate aggregated value. This function will also require metrics to store both the latest value and the corresponding timestamp.

Issue Type

kind/enhancement

Warning: Please confirm that this issue does not contain any sensitive information

  • I confirm this issue does not contain any sensitive information.
@yusong-yan yusong-yan added area/docdb YugabyteDB core features status/awaiting-triage Issue awaiting triage labels Dec 26, 2024
@yugabyte-ci yugabyte-ci added kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue labels Dec 26, 2024
yusong-yan added a commit that referenced this issue Dec 26, 2024
…etrics from Sum to Max

Summary:
`kSum` aggregation is currently used for many xCluster metrics, but this leads to overflows and inaccuracies for metrics like `last_read_hybridtime`, which should instead use maximum aggregation `kMax`.

Next Step [[ #25437 | #25437 ]]:
For optimal accuracy in the future, we should implement a `kLatest` aggregation function, which would require metric storing both the latest value and its corresponding timestamp.
Jira: DB-14654

Test Plan: Jenkins

Reviewers: hsunder, xCluster, jhe

Reviewed By: jhe

Subscribers: rthallam, ybase, ycdcxcluster, slingam

Differential Revision: https://phorge.dev.yugabyte.com/D40873
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/docdb YugabyteDB core features kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue status/awaiting-triage Issue awaiting triage
Projects
None yet
Development

No branches or pull requests

2 participants