DataSync location access test failed: could not perform S3:ListObjectsV2

[matt-challe]

Resource handler returned message:
"Invalid request provided: DataSync location access test failed: could not perform S3:ListObjectsV2 on bucket my_bucket Access denied. Ensure bucket access role has S3:ListBucket permission.

I am trying to create an S3 location for a DataSync Task and consistently running into the above error despite following AWS best practices . Note, S3:ListBucket is already given as a permission.

The error only occurs on "fresh" deploys where the stack is not already previously deployed successfully. My current workaround is to deploy granting the role S3 full access and then redeploy with the below policy. Below is the relevant portion of the python cdk code I am using.

Is this an AWS DataSync problem? Or a cdk bug?

def datasync_s3_access_role(bucket: s3.Bucket, role_name_suffix: str) -> iam.Role:
            
            # create bucket level policy
            datasync_s3_bucket_policy =iam.PolicyStatement(
                actions=[
                    "s3:GetBucketLocation",
                    "s3:ListBucket",
                    "s3:ListBucketMultipartUploads"
                ],
                resources=[bucket.bucket_arn],
                conditions={"StringEquals": {
                                    "aws:ResourceAccount": [self.account_full]
                    }}
                )
            
            # create object level policy
            datasync_s3_object_policy=iam.PolicyStatement(
                actions=[                
                    "s3:AbortMultipartUpload",
                    "s3:DeleteObject",
                    "s3:GetObject",
                    "s3:GetObjectTagging",
                    "s3:GetObjectVersion",
                    "s3:GetObjectVersionTagging",
                    "s3:ListMultipartUploadParts",
                    "s3:PutObject",
                    "s3:PutObjectTagging",
                ],
                resources=[f"{bucket.bucket_arn}/*"],
                conditions={"StringEquals": {
                        "aws:ResourceAccount": [self.account_full]
                    }}
            )
            
            # Create IAM role and policies for DataSync S3 access
            datasync_s3_access_role = iam.Role(self, 
                                                f"DataSyncRole-{role_name_suffix}-{env_config}",
                                                assumed_by=iam.ServicePrincipal("datasync.amazonaws.com"),
                                                role_name=f"DataSyncRole-{role_name_suffix}-{env_config}",)
            
            datasync_s3_access_role.add_to_policy(datasync_s3_bucket_policy)
            datasync_s3_access_role.add_to_policy(datasync_s3_object_policy)
            
            return datasync_s3_access_role
        
        
        datasync_s3_access_role_oci_to_aws = datasync_s3_access_role(oci_to_aws_bucket, 'oci-to-aws')
        datasync_s3_access_role_aws_to_oci = datasync_s3_access_role(aws_to_oci_bucket, 'aws-to-oci')

[StuBez]

I think the order of resource creation doesn't get correctly determined so the S3 access check is performed before the policy is attached to the role.

I was able to work around this by making the policy inline. I took the policy document from a default role that was available in the synthesised CloudFormation template.

DataSyncRoleDefaultPolicyCF9D0BD2:
    Type: AWS::IAM::Policy
    Properties:
      PolicyDocument:
        Statement:
          - Action:
              - s3:Abort*
              - s3:DeleteObject*
              - s3:GetBucket*
              - s3:GetBucketLocation
              - s3:GetObject
              - s3:GetObject*
              - s3:GetObjectTagging
              - s3:List*
              - s3:ListBucket
              - s3:ListObjectsV2
              - s3:PutObject
              - s3:PutObjectAcl
              - s3:PutObjectLegalHold
              - s3:PutObjectRetention
              - s3:PutObjectTagging
              - s3:PutObjectVersionTagging
            Effect: Allow
            Resource:
              - Fn::ImportValue: otto-core:ExportsOutputFnGetAttBotArtifactsBF93467EArn6A5C860D
              - Fn::Join:
                  - ""
                  - - Fn::ImportValue: otto-core:ExportsOutputFnGetAttBotArtifactsBF93467EArn6A5C860D
                    - /*
          - Action:
              - elasticfilesystem:ClientMount
              - elasticfilesystem:ClientRead
            Effect: Allow
            Resource:
              Fn::ImportValue: otto-core:ExportsOutputFnGetAttTaskTemporaryFSE066DEF0ArnF57FF42E
        Version: "2012-10-17"
      PolicyName: DataSyncRoleDefaultPolicyCF9D0BD2
      Roles:
        - Ref: DataSyncRole90D44C0A
    Metadata:
      aws:cdk:path: otto-data-sync/DataSyncRole/DefaultPolicy/Resource

datasync_role = iam.Role(
        self,
        "DataSyncRole",
        assumed_by=iam.ServicePrincipal("datasync.amazonaws.com"),
        # This policy has to be inline due to a race condition with the s3_location
        inline_policies={
            "DataSyncS3Policy": iam.PolicyDocument(
                statements=[
                    iam.PolicyStatement(
                        actions=[
                            "s3:Abort*",
                            "s3:DeleteObject*",
                            "s3:GetBucket*",
                            "s3:GetBucketLocation",
                            "s3:GetObject",
                            "s3:GetObject*",
                            "s3:GetObjectTagging",
                            "s3:List*",
                            "s3:ListBucket",
                            "s3:ListObjectsV2",
                            "s3:PutObject",
                            "s3:PutObjectAcl",
                            "s3:PutObjectLegalHold",
                            "s3:PutObjectRetention",
                            "s3:PutObjectTagging",
                            "s3:PutObjectVersionTagging"
                        ],
                        effect=iam.Effect.ALLOW,
                        resources=[
                            core_stack.artifact_bucket.bucket_arn,
                            f"{core_stack.artifact_bucket.bucket_arn}/*"
                        ]
                    )
                ]
            )
        }
    )
    ```