Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for restoring from snapshot with search replicas #16111

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

vinaykpud
Copy link

Description

In this PR we are adding,

  1. Support for restoring search replicas from a snapshot taken with search replica
  2. Add validations to ensure compatibility between DOCUMENT and SEGMENT replication type while snapshot restoration. Following are the allowed rules:
  • Snapshots taken when the replication type is DOCUMENT can be restored to DOCUMENT or SEGMENT types. If we are restoring to SEGMENT type more search replicas can be added.
  • Snapshots taken when the replication type is SEGMENT and without any search replicas can be restored to DOCUMENT or SEGMENT types. If we are restoring to SEGMENT type more search replicas can be added.
  • Snapshots taken when the replication type is SEGMENT and with search replicas can be restored to DOCUMENT or SEGMENT types. If we are restoring to SEGMENT type more search replicas can be added. If we are restoring to DOCUMENT type search replica count must be set to 0 in the settings.

Related Issues

Resolves #15532
Related #15306

Check List

  • Functionality includes testing.
  • API changes companion pull request created, if applicable.
  • Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Copy link
Contributor

❌ Gradle check result for 7556720: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@@ -650,6 +661,23 @@ public ClusterState execute(ClusterState currentState) {
return allocationService.reroute(updatedState, "restored snapshot [" + snapshot + "]");
}

private void validateReplicationTypeRestoreSettings(String snapshotReplicationType,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpicks -

Lets pass a ReplicationType here and parse in the caller and use ReplicationType. DOCUMENT.equals(replicationType)

is snapshotIndexMetadata the metadata inside the snapshot? Passing that as restoreReplicationType is a bit confusing. How about snapshottedMetadata and updatedMetadata ?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

snapshotIndexMetadata contains metadata of snapshot and whatever setting user is trying to override. So basically its a metadata which will be applied for resulting restored index.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you mean like this?

private void validateReplicationTypeRestoreSettings(IndexMetadata snapshottedMetadata, IndexMetadata updatedMetadata)

Copy link
Contributor

github-actions bot commented Oct 2, 2024

❌ Gradle check result for 7f03249: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

github-actions bot commented Oct 2, 2024

❌ Gradle check result for a84f849: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@vinaykpud vinaykpud closed this Oct 2, 2024
@vinaykpud vinaykpud reopened this Oct 2, 2024
Copy link
Contributor

github-actions bot commented Oct 2, 2024

❌ Gradle check result for a84f849: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

github-actions bot commented Oct 2, 2024

❌ Gradle check result for 53821df: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

github-actions bot commented Oct 2, 2024

❌ Gradle check result for ebccad3: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

github-actions bot commented Oct 3, 2024

❌ Gradle check result for b757f29: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

github-actions bot commented Oct 3, 2024

❌ Gradle check result for 7f79749: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@vinaykpud vinaykpud closed this Oct 3, 2024
@vinaykpud vinaykpud reopened this Oct 3, 2024
Vinay Krishna Pudyodu added 5 commits October 3, 2024 10:07
Signed-off-by: Vinay Krishna Pudyodu <[email protected]>
Signed-off-by: Vinay Krishna Pudyodu <[email protected]>
Signed-off-by: Vinay Krishna Pudyodu <[email protected]>
Signed-off-by: Vinay Krishna Pudyodu <[email protected]>
Copy link
Contributor

github-actions bot commented Oct 3, 2024

✅ Gradle check result for 641a804: SUCCESS

Copy link

codecov bot commented Oct 3, 2024

Codecov Report

Attention: Patch coverage is 63.63636% with 4 lines in your changes missing coverage. Please review.

Project coverage is 71.84%. Comparing base (6020c58) to head (641a804).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
.../java/org/opensearch/snapshots/RestoreService.java 50.00% 2 Missing and 2 partials ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main   #16111      +/-   ##
============================================
- Coverage     71.88%   71.84%   -0.05%     
+ Complexity    64496    64475      -21     
============================================
  Files          5291     5291              
  Lines        301668   301679      +11     
  Branches      43576    43579       +3     
============================================
- Hits         216863   216747     -116     
- Misses        67031    67123      +92     
- Partials      17774    17809      +35     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@vinaykpud vinaykpud marked this pull request as ready for review October 3, 2024 21:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
v2.18.0 Issues and PRs related to version 2.18.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[RW Separation] Updates to restore from snapshot with added search replicas
2 participants