Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure that all cluster nodes have valid metadata in the cluster topology seen by the proxy #92

Open
alicel opened this issue Jan 9, 2023 · 0 comments

Comments

@alicel
Copy link
Collaborator

alicel commented Jan 9, 2023

This is an additional safety check that stems from observing a somewhat brittle behaviour of the cluster metadata in early C* 2.1 versions. While investigating the issue in #91, we noticed that the proxy does not validate that the addresses of all cluster nodes are correctly populated.

If the topology metadata in the system tables of a cluster is malformed or incomplete, the proxy builds its topology view regardless and then tries to use potentially invalid assigned hosts (e.g. hosts missing the address) when establishing connections to the cluster. This of course results in a failure which affects the application and becomes a blocker for a migration.

We should add reasonable checks to validate that the data of each assigned host is complete, so that we know that it can be used without issues. If we detect incomplete or invalid metadata (and no fallback or remediation is possible), we should prevent the proxy from starting and display a clear message in the logs. This may sound drastic, but ignoring some hosts or silently working around this in some other suboptimal way would be unreliable, hiding problems that may become apparent only when the migration gets underway.

┆Issue is synchronized with this Jira Task by Unito
┆Components: Proxy
┆Issue Number: ZDM-512

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant