implement user-configurable retry classifiers #2977

Velfi · 2023-09-08T20:58:06Z

Read the RFC here

Motivation and Context

#2417

Description

Exactly what it says on the tin. I have a related RFC to publish that goes into more depth.

Testing

I wrote an integration test that ensures a custom retry classifier can be set and is called.

Checklist

I have updated CHANGELOG.next.toml if I made changes to the smithy-rs codegen or runtime crates
I have updated CHANGELOG.next.toml if I made changes to the AWS SDK, generated SDK code, or SDK runtime crates

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

github-actions · 2023-09-08T21:26:25Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
Client Test (ignoring whitespace)
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

rcoh

If I'm reading this PR correctly, all (?) of our built-in classifiers are actually incorrect since they ignore the previous result causing retries to be blown away.

I think this is probably a hint that this interface as written is a tripping hazard / face rake and we should try to revise the implementor experience to make it easier to do the right thing—possibly by just altering the behavior of the main loop to not replace Some with None?

Can dig in more in the text based RFC

rcoh · 2023-09-11T13:43:36Z

aws/sdk/integration-tests/s3/tests/retry-classifier-customization.rs

+impl ClassifyRetry for CustomRetryClassifier {
+    fn classify_retry(
+        &self,
+        _: &InterceptorContext,
+        _: Option<RetryClassifierResult>,
+    ) -> Option<RetryClassifierResult> {
+        *self.counter.lock().unwrap() += 1;
+
+        Some(RetryClassifierResult::DontRetry)
+    }
+
+    fn name(&self) -> &'static str {
+        "Custom Retry Classifier"
+    }
+}


recommendation: Actually look at the request/response, & status code in this integration test to ensure that everything is properly wired.

aws/sdk/integration-tests/s3/tests/retry-classifier-customization.rs

...mazon/smithy/rust/codegen/client/smithy/customizations/RetryClassifierConfigCustomization.kt

rcoh · 2023-09-11T13:47:00Z

...mazon/smithy/rust/codegen/client/smithy/customizations/RetryClassifierConfigCustomization.kt

+                    pub fn retry_classifiers(&self) -> impl Iterator<Item = #{SharedRetryClassifier}> + '_ {
+                        self.runtime_components.retry_classifiers()


is there an existing (customer usable) piece of code somewhere to classify a retry given a list of classifiers? Should this expose a list of classifiers or should it unify the list (effectively hiding the behavior that there is a list of these things)

I chose to return an iterator here because that's what we're doing for other 'list collected' things like interceptors.

rust-runtime/aws-smithy-runtime-api/src/client/retries/classifiers.rs

rust-runtime/aws-smithy-runtime/src/client/connectors/connection_poisoning.rs

rust-runtime/aws-smithy-runtime/src/client/retries/classifiers.rs

github-actions · 2023-10-04T17:01:23Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
Client Test (ignoring whitespace)
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

github-actions · 2023-10-04T19:11:30Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
Client Test (ignoring whitespace)
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

…sifiers fix incorrect test JSON names add missing HttpStatusCodeClassifier to AssumeRoleProvider add section helper fn registerRetryClassifier to ServiceRuntimePluginSection.RegisterRuntimeComponents remove From<*Classifier> impls in favor of IntoShared

github-actions · 2023-10-05T19:35:21Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
Client Test (ignoring whitespace)
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

rename RetryClassifierResult to RetryAction

github-actions · 2023-10-05T21:32:23Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
Client Test (ignoring whitespace)
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

github-actions · 2023-10-06T15:27:52Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
Client Test (ignoring whitespace)
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

aws/rust-runtime/aws-config/src/http_credential_provider.rs

aws/rust-runtime/aws-runtime/src/retries.rs

aws/rust-runtime/aws-runtime/src/retries/classifiers.rs

aws/sdk-codegen/src/main/kotlin/software/amazon/smithy/rustsdk/HttpResponseChecksumDecorator.kt

jdisanti · 2023-10-06T17:51:19Z

...mazon/smithy/rust/codegen/client/smithy/customizations/RetryClassifierConfigCustomization.kt

+                        pub fn push_retry_classifier(&mut self, retry_classifier: #{SharedRetryClassifier}) -> &mut Self {
+                            self.runtime_components.push_retry_classifier(retry_classifier);


IntoShared will be merged into main shortly:

Suggested change

pub fn push_retry_classifier(&mut self, retry_classifier: #{SharedRetryClassifier}) -> &mut Self {

self.runtime_components.push_retry_classifier(retry_classifier);

pub fn push_retry_classifier(&mut self, retry_classifier: impl #{RetryClassifier} + 'static) -> &mut Self {

self.runtime_components.push_retry_classifier(#{IntoShared}::into_shared(retry_classifier));

Although, why do we have both retry_classifier and push_retry_classifier?

I was trying to mirror the Interceptor setters API.

rust-runtime/aws-smithy-runtime-api/src/client/retries/classifiers.rs

rust-runtime/aws-smithy-runtime-api/src/client/runtime_components.rs

rust-runtime/aws-smithy-runtime/src/client/orchestrator/operation.rs

rust-runtime/aws-smithy-runtime/src/client/retries/strategy/fixed_delay.rs

Velfi · 2023-10-06T18:10:14Z

...mazon/smithy/rust/codegen/client/smithy/customizations/RetryClassifierConfigCustomization.kt

+                        """
+                        /// Add type implementing [`ClassifyRetry`](#{ClassifyRetry}) that will be used by the
+                        /// [`RetryStrategy`](#{RetryStrategy}) to determine what responses should be retried.
+                        ///
+                        /// A retry classifier configured by this method will run according to its priority.
+                        ///
+                        /// ## Examples
+                        /// ```no_run
+                        /// ## ##[cfg(test)]
+                        /// ## mod tests {
+                        /// ## ##[test]
+                        /// ## fn example() {
+                        /// ## }
+                        /// ## }
+                        /// ```
+                        pub fn retry_classifier(mut self, retry_classifier: impl #{ClassifyRetry} + 'static) -> Self {
+                            self.push_retry_classifier(#{SharedRetryClassifier}::new(retry_classifier));
+                            self
+                        }
+
+                        /// Add a [`SharedRetryClassifier`](#{SharedRetryClassifier}) that will be used by the
+                        /// [`RetryStrategy`](#{RetryStrategy}) to determine what responses should be retried.
+                        ///
+                        /// A retry classifier configured by this method will run according to its priority.
+                        ///
+                        /// ## Examples
+                        /// ```no_run
+                        /// ## ##[cfg(test)]
+                        /// ## mod tests {
+                        /// ## ##[test]
+                        /// ## fn example() {
+                        /// ## }
+                        /// ## }
+                        /// ```
+                        pub fn push_retry_classifier(&mut self, retry_classifier: #{SharedRetryClassifier}) -> &mut Self {


TODO add examples

refactor RetryAction remove previous_action arg from ClassifyRetry::classify_retry

github-actions · 2023-10-06T21:10:27Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
Client Test (ignoring whitespace)
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

github-actions · 2023-10-09T15:03:44Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
Client Test (ignoring whitespace)
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

fix incorrect XML in test remove unused dependency

github-actions · 2023-10-09T16:28:04Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
Client Test (ignoring whitespace)
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

github-actions · 2023-10-09T16:53:02Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
Client Test (ignoring whitespace)
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

rcoh

Overall looks great. My only top-level thought to consider is that we may want to split DontCare out from the list of RetryActions since that's something only a classifier cares about (and the orchestrator doesn't really).

Thinking something like:

enum RetryAction {
  ActionIndicated(RetryReason),
  NoActionIndicated
}

or something. This is definitely not critical though.

aws/rust-runtime/aws-config/src/imds/client.rs

aws/rust-runtime/aws-config/src/profile/credentials/exec.rs

rcoh · 2023-10-09T17:55:12Z

aws/sdk-codegen/src/main/kotlin/software/amazon/smithy/rustsdk/RetryClassifierDecorator.kt

@@ -24,43 +25,49 @@ class RetryClassifierDecorator : ClientCodegenDecorator {
        baseCustomizations: List<OperationCustomization>,
    ): List<OperationCustomization> = baseCustomizations +
        OperationRetryClassifiersFeature(codegenContext, operation)
+
+    override fun serviceRuntimePluginCustomizations(


no actual changes behaviorally to this file, right?

It used to mount all 5 classifiers. Now it only mounts the two that need knowledge of a specific error type to work.

Scratch that, I combined the AWS classifiers because of the new "retry after" handling. This is now covered by the AWS Error code classifier

...egration-tests/s3/tests/data/request-information-headers/three-retries_and-then-success.json

rcoh · 2023-10-09T17:59:59Z

...kotlin/software/amazon/smithy/rust/codegen/client/smithy/customize/RequiredCustomizations.kt

@@ -49,15 +52,17 @@ class RequiredCustomizations : ClientCodegenDecorator {
        baseCustomizations +
            MetadataCustomization(codegenContext, operation) +
            IdempotencyTokenGenerator(codegenContext, operation) +
-            HttpChecksumRequiredGenerator(codegenContext, operation)
+            HttpChecksumRequiredGenerator(codegenContext, operation) +
+            RetryClassifierOperationCustomization(codegenContext, operation)


how did this work before?

Previously, all retry classifiers were set per-operation in the RetryClassifierDecorator. Now, they can be set at the config level AND the operation level.

...software/amazon/smithy/rust/codegen/client/smithy/generators/client/FluentClientGenerator.kt

rcoh · 2023-10-09T18:06:47Z

rust-runtime/aws-smithy-runtime/src/client/retries/classifiers.rs

+
+    /// Return the priority of this retry classifier.
+    pub fn priority() -> RetryClassifierPriority {
+        RetryClassifierPriority::modeled_as_retryable_classifier()


consider renaming this something like RetryTraitClassifier (non blocking)

rust-runtime/aws-smithy-runtime/src/client/retries/classifiers.rs

rust-runtime/aws-smithy-runtime-api/src/client/retries/classifiers.rs

github-actions · 2023-10-09T21:25:47Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
Client Test (ignoring whitespace)
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

github-actions · 2023-10-10T14:54:40Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
Client Test (ignoring whitespace)
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

github-actions · 2023-10-10T15:58:43Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
Client Test (ignoring whitespace)
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

add CHANGELOG entry

github-actions · 2023-10-10T16:33:43Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
Client Test (ignoring whitespace)
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

github-actions · 2023-10-10T18:59:36Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
Client Test (ignoring whitespace)
No codegen difference in the Server Test
No codegen difference in the Server Test Python
No codegen difference in the Server Test Typescript

A new doc preview is ready to view.

implement user-configurable retry classifiers

7c4a63f

Velfi requested review from a team as code owners September 8, 2023 20:58

rcoh requested changes Sep 11, 2023

View reviewed changes

Merge branch 'main' into zhessler-user-configurable-retry-classifiers

d994100

Velfi added 2 commits October 4, 2023 13:27

fix codegen test

c8c05d4

fix KMS integration test

425e5f4

Velfi added 2 commits October 5, 2023 13:34

fix test_retry_classifier_customization

862d79d

Velfi added 3 commits October 5, 2023 15:42

rename SmithyErrorClassifier to TransientErrorClassifier

b6bd15a

rename RetryClassifierResult to RetryAction

fix unused import warning in codegen

ee3707a

fix more tests

f8abd87

Velfi added 4 commits October 6, 2023 09:32

fix another test

e2da8a1

Merge branch 'main' into zhessler-user-configurable-retry-classifiers

cfea5f4

rename RetryAction variants

f6be431

fix retry action docs

2baefa4

jdisanti reviewed Oct 6, 2023

View reviewed changes

Velfi commented Oct 6, 2023

View reviewed changes

Velfi added 5 commits October 6, 2023 13:45

respond to PR comments

a5a94d8

Merge branch 'main' into zhessler-user-configurable-retry-classifiers

10aca1b

fix test broken by merge

32ee53f

remove the unused FixedDelayRetryStrategy

55e2a5c

flip RetryClassifierPriority

8598891

refactor RetryAction remove previous_action arg from ClassifyRetry::classify_retry

fix missing classifier variant in standard retry strategy

0be791d

Velfi added the breaking-change This will require a breaking change label Oct 9, 2023

fix sneaky extra writable

2f221c6

fix incorrect XML in test remove unused dependency

remove unused import

c41317e

rcoh approved these changes Oct 9, 2023

View reviewed changes

Velfi added 2 commits October 9, 2023 15:57

refactor retry actions and reasons

33900bd

Merge branch 'main' into zhessler-user-configurable-retry-classifiers

9e4ab0f

Velfi added 2 commits October 10, 2023 09:36

fix test

28df537

Merge branch 'main' into zhessler-user-configurable-retry-classifiers

e406aa1

add docs to generated retry classifier setters

e8b28b0

Velfi added 2 commits October 10, 2023 11:15

add retry classifier config override test

76b2b8e

add CHANGELOG entry

Merge branch 'main' into zhessler-user-configurable-retry-classifiers

2402d1c

ysaito1001 approved these changes Oct 10, 2023

View reviewed changes

Merge branch 'main' into zhessler-user-configurable-retry-classifiers

918a2db

Velfi enabled auto-merge October 10, 2023 18:43

Velfi added this pull request to the merge queue Oct 10, 2023

Merged via the queue into main with commit 96a9f84 Oct 10, 2023
39 of 41 checks passed

Velfi deleted the zhessler-user-configurable-retry-classifiers branch October 10, 2023 19:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

implement user-configurable retry classifiers #2977

implement user-configurable retry classifiers #2977

Velfi commented Sep 8, 2023 •

edited

Loading

github-actions bot commented Sep 8, 2023

rcoh left a comment

rcoh Sep 11, 2023

rcoh Sep 11, 2023

Velfi Oct 5, 2023

github-actions bot commented Oct 4, 2023

github-actions bot commented Oct 4, 2023

github-actions bot commented Oct 5, 2023

github-actions bot commented Oct 5, 2023

github-actions bot commented Oct 6, 2023

jdisanti Oct 6, 2023

Velfi Oct 6, 2023 •

edited

Loading

Velfi Oct 6, 2023

github-actions bot commented Oct 6, 2023

github-actions bot commented Oct 9, 2023

github-actions bot commented Oct 9, 2023

github-actions bot commented Oct 9, 2023

rcoh left a comment

rcoh Oct 9, 2023

Velfi Oct 9, 2023

Velfi Oct 9, 2023

rcoh Oct 9, 2023

Velfi Oct 9, 2023

rcoh Oct 9, 2023

github-actions bot commented Oct 9, 2023

github-actions bot commented Oct 10, 2023

github-actions bot commented Oct 10, 2023

github-actions bot commented Oct 10, 2023

github-actions bot commented Oct 10, 2023

		pub fn retry_classifiers(&self) -> impl Iterator<Item = #{SharedRetryClassifier}> + '_ {
		self.runtime_components.retry_classifiers()

		pub fn push_retry_classifier(&mut self, retry_classifier: #{SharedRetryClassifier}) -> &mut Self {
		self.runtime_components.push_retry_classifier(retry_classifier);

implement user-configurable retry classifiers #2977

implement user-configurable retry classifiers #2977

Conversation

Velfi commented Sep 8, 2023 • edited Loading

Motivation and Context

Description

Testing

Checklist

github-actions bot commented Sep 8, 2023

rcoh left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Oct 4, 2023

github-actions bot commented Oct 4, 2023

github-actions bot commented Oct 5, 2023

github-actions bot commented Oct 5, 2023

github-actions bot commented Oct 6, 2023

Choose a reason for hiding this comment

Velfi Oct 6, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Oct 6, 2023

github-actions bot commented Oct 9, 2023

github-actions bot commented Oct 9, 2023

github-actions bot commented Oct 9, 2023

rcoh left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Oct 9, 2023

github-actions bot commented Oct 10, 2023

github-actions bot commented Oct 10, 2023

github-actions bot commented Oct 10, 2023

github-actions bot commented Oct 10, 2023

Velfi commented Sep 8, 2023 •

edited

Loading

Velfi Oct 6, 2023 •

edited

Loading