AssumeRoleAWSCredentials can not be configured to use regional endpoints

[airxhi]

Describe the bug

When instantiating a new instance of AssumeRoleAWSCredentials, there is no option to configure the Security Token Service client to use regional endpoints for performing the API call.

This then requires any compute instance that makes use of these credentials to allow outbound to the us-east-1 Amazon ranges to enable assuming role using this method.

Otherwise the GenerateNewCredentials method will error.

Expected Behavior

An option to specify that the sts client should use the sts regional endpoints.

Then the GenerateNewCredentials method would not time out when the global endpoint was not accessible, but a regional endpoint was.

Current Behavior

Assume role request times out, as it cannot reach the global sts endpoint in the us-east-1 region.

Amazon.Runtime.AmazonClientException: Error calling AssumeRole for role <REDACTED> ---> Amazon.Runtime.AmazonServiceException: A WebException with status ConnectFailure was thrown. ---> System.Net.WebException: Unable to connect to the remote server ---> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 52.119.198.216:443

Reproduction Steps

Run the following on a compute instance in a VPC with no internet access in the eu-west-1 region, with a VPC endpoint for sts.eu-west-1.amazonaws.com also deployed in the VPC.

var creds = new BasicAWSCredentials("ACCESSKEY", "SECRETKEY");

var newCreds = new AssumeRoleAWSCredentials(creds, "ROLE_ARN", "ROLE_SESSION_NAME", new AssumeRoleAWSCredentialsOptions());

var immutableCreds = newCreds.GetCredentials(); // This will timeout, as it will try to access the us-east-1 sts endpoint

Possible Solution

Add an additional property to AssumeRoleAWSCredentialsOptions.cs to configure the sts client to use regional endpoints.
https://github.com/aws/aws-sdk-net/blob/main/sdk/src/Core/Amazon.Runtime/Credentials/AssumeRoleAWSCredentialsOptions.cs

This property should then be checked in the GenerateNewCredentials function in AssumeRoleAWSCredentials.cs and should set the property on the AmazonSecurityTokenServiceConfig object that is created before being passed to the STS client.
https://github.com/aws/aws-sdk-net/blob/main/sdk/src/Core/Amazon.Runtime/Credentials/AssumeRoleAWSCredentials.cs

Additional Information/Context

No response

AWS .NET SDK and/or Package version used

AWSSDK.Core 3.7.300.56

Targeted .NET Platform

.NET Framework 4.7.2

Operating System and version

Windows Server 2019

[ashishdhingra]

@airxhi Good morning. Thanks for opening the issue. Please advise if you have tried guidance in AWS STS Regionalized endpoints for using regional endpoints for STS. I guess this functionality is consistent across all AWS SDKs and if any of the other SDK has any option as proposed by you in the issue description.

Thanks,
Ashish

[airxhi]

Hi @ashishdhingra !

Please advise if you have tried guidance in AWS STS Regionalized endpoints for using regional endpoints for STS.

We currently use the regionalized endpoints with the AmazonSecurityTokenServiceClient and manually handle the assume role calls and have implemented our own refreshing system. However, our system is subpar to using the AssumeRoleAWSCredentials and we would much prefer to use the native AWS SDK implementation rather than rolling our own.

For example, our current implementation inside our own system:

[Deprecated]
// This is our old, but working system
private AmazonSecurityTokenServiceClient GetSecurityTokenServiceClient(string region)
{
    if (_securityTokenServiceClient == null)
    {
        _securityTokenServiceClient = new AWSClient<AmazonSecurityTokenServiceClient>(
            this, 
            (a, b) => new AmazonSecurityTokenServiceClient(a, new AmazonSecurityTokenServiceConfig()
            {
                 RegionEndpoint = b,
                 StsRegionalEndpoints = StsRegionalEndpointsValue.Regional // <- enabling using regional endpoints
            }),
            (a) => new AmazonSecurityTokenServiceClient(new AmazonSecurityTokenServiceConfig()
            {
                RegionEndpoint = a,
                StsRegionalEndpoints = StsRegionalEndpointsValue.Regional // <- enabling using regional endpoints
            }));
    }
    return _securityTokenServiceClient.GetClient(region);
}

I'm not aware of any other SDK's implementation of the equivalent, boto3 does not have this capability as they don't have an equivalent AssumeRoleCredentials object if I am reading their docs correctly.

The Java SDK has an equivalent builder that allows the caller to supply an sts client with the region endpoints enabled, this approach would also work for the .NET SDK. (https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/STSAssumeRoleSessionCredentialsProvider.Builder.html)

[ashishdhingra]

Needs review with the team. Most likely a feature request since the behavior is implemented per documentation. Java SDK allows to use STSClient as per last comment. This should perhaps be implemented consistently across all SDK(s) and documented accordingly.

[airxhi]

Makes sense!

The reason I raised this as a bug is that the implementation of GenerateNewCredentials is as follows and is clearly trying to support regional endpoints but is just being ignored as the STS client doesn't care unless the config is configured with the STSRegionalEndpoints option.

Below code is all from https://github.com/aws/aws-sdk-net/blob/main/sdk/src/Core/Amazon.Runtime/Credentials/AssumeRoleAWSCredentials.cs

You can see in the below that the region is fetched:

var region = FallbackRegionFactory.GetRegionEndpoint() ?? DefaultSTSClientRegion;

and then configured on STS client config

stsConfig.RegionEndpoint = region;

but this is just ignored unless the STSRegionalEndpoints property is configured.

protected override CredentialsRefreshState GenerateNewCredentials()
{
    var region = FallbackRegionFactory.GetRegionEndpoint() ?? DefaultSTSClientRegion;
    ICoreAmazonSTS coreSTSClient = GlobalRuntimeDependencyRegistry.Instance.GetInstance<ICoreAmazonSTS>(ServiceClientHelpers.STS_ASSEMBLY_NAME, ServiceClientHelpers.STS_SERVICE_CLASS_NAME, 
        new CreateInstanceContext(new SecurityTokenServiceClientContext {Action = SecurityTokenServiceClientContext.ActionContext.AssumeRoleAWSCredentials, Region = region, ProxySettings = Options?.ProxySettings } ));

    if (coreSTSClient == null)
    {
        try
        {
            var stsConfig = ServiceClientHelpers.CreateServiceConfig(ServiceClientHelpers.STS_ASSEMBLY_NAME, ServiceClientHelpers.STS_SERVICE_CONFIG_NAME);
            stsConfig.RegionEndpoint = region;

            if (Options?.ProxySettings != null)
            {
                stsConfig.SetWebProxy(Options.ProxySettings);
            }

            coreSTSClient = ServiceClientHelpers.CreateServiceFromAssembly<ICoreAmazonSTS>(
                ServiceClientHelpers.STS_ASSEMBLY_NAME, ServiceClientHelpers.STS_SERVICE_CLASS_NAME, SourceCredentials, stsConfig);
        }
        catch (Exception e)
        {
            if (InternalSDKUtils.IsRunningNativeAot())
            {
                throw new MissingRuntimeDependencyException(ServiceClientHelpers.STS_ASSEMBLY_NAME, ServiceClientHelpers.STS_SERVICE_CLASS_NAME, nameof(GlobalRuntimeDependencyRegistry.RegisterSecurityTokenServiceClient));
            }

            var msg = string.Format(CultureInfo.CurrentCulture,
                "Assembly {0} could not be found or loaded. This assembly must be available at runtime to use Amazon.Runtime.AssumeRoleAWSCredentials.",
                ServiceClientHelpers.STS_ASSEMBLY_NAME);
            var exception = new InvalidOperationException(msg, e);
            Logger.GetLogger(typeof(AssumeRoleAWSCredentials)).Error(exception, exception.Message);
            throw exception;
        }
    }



    var credentials = coreSTSClient.CredentialsFromAssumeRoleAuthentication(RoleArn, RoleSessionName, Options);
    _logger.InfoFormat("New credentials created for assume role that expire at {0}", credentials.Expiration.ToString("yyyy-MM-ddTHH:mm:ss.fffffffK", CultureInfo.InvariantCulture));
    return new CredentialsRefreshState(credentials, credentials.Expiration);
}

[airxhi]

Hi! We have implemented a fix internally via the following, this is not a suggestion on how to solve the problem/feature in the SDK, but it's the functionality that we needed! Hope this helps! :)

internal class AssumeRoleAWSCredentialsRegionalOptions : AssumeRoleAWSCredentialsOptions
{
    public RegionEndpoint RegionEndpoint { get; set; }
    public StsRegionalEndpointsValue STSRegionalEndpoints { get; set; }
}

internal class AssumeRoleAWSCredentialsRegional : AssumeRoleAWSCredentials
{
    private RegionEndpoint DefaultSTSClientRegion = RegionEndpoint.EUWest1;
    public new AssumeRoleAWSCredentialsRegionalOptions Options { get; private set; }
    
    protected override CredentialsRefreshState GenerateNewCredentials()
    {
        var regionEndpoint = FallbackRegionFactory.GetRegionEndpoint() ?? DefaultSTSClientRegion;
        ICoreAmazonSTS serviceFromAssembly;
        try
        {
            var serviceConfig = ServiceClientHelpers.CreateServiceConfig("AWSSDK.SecurityToken", "Amazon.SecurityToken.AmazonSecurityTokenServiceConfig");
            serviceConfig.RegionEndpoint = regionEndpoint;
            if (Options?.ProxySettings != null) { // disable web proxy settings as we don't use them and causing issues.
                serviceConfig.SetWebProxy(Options.ProxySettings);
            }
            if (Options?.RegionEndpoint != null)
            {
                serviceConfig.RegionEndpoint = Options.RegionEndpoint;
            }
            if (Options?.STSRegionalEndpoints != null)
            {
                ((AmazonSecurityTokenServiceConfig)serviceConfig).StsRegionalEndpoints =
                    Options.STSRegionalEndpoints;
            }
            serviceFromAssembly = ServiceClientHelpers.CreateServiceFromAssembly<ICoreAmazonSTS>("AWSSDK.SecurityToken", "Amazon.SecurityToken.AmazonSecurityTokenServiceClient", SourceCredentials, serviceConfig);
        }
        catch (Exception ex)
        {
            var operationException = new InvalidOperationException(string.Format(CultureInfo.CurrentCulture, "Assembly {0} could not be found or loaded. This assembly must be available at runtime to use Amazon.Runtime.AssumeRoleAWSCredentials.", (object) "AWSSDK.SecurityToken"), ex);
            Logger.GetLogger(typeof (AssumeRoleAWSCredentials)).Error(operationException, operationException.Message, Array.Empty<object>());
            throw operationException;
        }
        var credentials = serviceFromAssembly.CredentialsFromAssumeRoleAuthentication(RoleArn, RoleSessionName, this.Options);
        {
            credentials.Expiration.ToString("yyyy-MM-ddTHH:mm:ss.fffffffK", CultureInfo.InvariantCulture);
        }
        return new CredentialsRefreshState(credentials, credentials.Expiration);
    }

    public AssumeRoleAWSCredentialsRegional(AWSCredentials sourceCredentials, string roleArn, string roleSessionName) : base(sourceCredentials, roleArn, roleSessionName)
    {
    }

    public AssumeRoleAWSCredentialsRegional(AWSCredentials sourceCredentials, string roleArn, string roleSessionName, AssumeRoleAWSCredentialsRegionalOptions options) : base(sourceCredentials, roleArn, roleSessionName, options)
    {
        Options = options;
    }
}

[dscpinheiro]

Thanks for the suggestion. I know it doesn't help your use case right now, but in the next version of the SDK (V4) we're getting rid of the StsRegionalEndpointsValue enum (all STS calls will be made to the regional endpoints by default).

You can read more about V4 in #3362 (this is also where we'll announce when the preview is available).

[github-actions]

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.