Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pageserver: single per-timeline variable backpressure signal #10116

Open
Tracked by #8390
erikgrinaker opened this issue Dec 12, 2024 · 0 comments
Open
Tracked by #8390

pageserver: single per-timeline variable backpressure signal #10116

erikgrinaker opened this issue Dec 12, 2024 · 0 comments
Assignees
Labels
a/performance Area: relates to performance of the system c/storage/pageserver Component: storage: pageserver

Comments

@erikgrinaker
Copy link
Contributor

erikgrinaker commented Dec 12, 2024

Currently, the Pageserver backpressures the compute by sending a set of LSNs and having the compute calculate the backpressure. This has a few issues:

  • Computes are upgraded/restarted rarely, so the computation can't easily be changed.
  • Protocol changes are tricky due to backwards compatibility.
  • The backpressure is binary (no backpressure, or sleep between each operation).

Instead, we should consider exposing a single backpressure signal to the compute: an integer specifying a bytes/second WAL append rate. When the compute appends to the WAL, it sleeps as: sleep(size / rate) -- we can optionally subtract the IO time if we want to have a more accurate rate. A rate of 0 means a full stall, a rate of <0 means no throttling. Different shards will compute different target rates, which should be taken into account.

We could also simply do the backpressure on the Safekeeper, but that could allow the compute to build up a large WAL backlog locally.

@erikgrinaker erikgrinaker added a/performance Area: relates to performance of the system c/storage/pageserver Component: storage: pageserver labels Dec 12, 2024
@erikgrinaker erikgrinaker self-assigned this Dec 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a/performance Area: relates to performance of the system c/storage/pageserver Component: storage: pageserver
Projects
None yet
Development

No branches or pull requests

1 participant