Random failures with aws sync

[Jourdan-He]

Hi,

I am having issues with file uploading from my institution to aws. An example command I used is:
aws s3 sync --no-follow-symlinks --only-show-errors --exclude 'download*' --exclude '**/.[a-z]*' --exclude '**/.[A-Z]*' --exclude '**/._*' --delete /storage1/fs1/rvmartin/Active/GEOS-Chem-shared/ExtData/GEOSCHEM_RESTARTS s3://gcgrid/GEOSCHEM_RESTARTS
My s3 bucket permission is

Some errors looks like this:
upload failed: GEOSCHEM_RESTARTS/GC_14.0.0/GEOSChem.Restart.fullchem.20190701_0000z.c360.nc4 to s3://gcgrid/GEOSCHEM_RESTARTS/GC_14.0.0/GEOSChem.Restart.fullchem.20190701_0000z.c360.nc4 An error occurred (AccessDenied) when calling the CreateMultipartUpload operation: Access Denied
upload failed: GEOSCHEM_RESTARTS/v2016-07/initial_GEOSChem_rst.4x5_TOMAS15.nc to s3://gcgrid/GEOSCHEM_RESTARTS/v2016-07/initial_GEOSChem_rst.4x5_TOMAS15.nc An error occurred (AccessDenied) when calling the UploadPart operation: Access Denied

The strange point is that some files I can upload successfully, and some failed uploads still show up on aws, I am really confused.
The version of aws-cli I used is 2.8.13

[RyanFitzSimmonsAK]

Hi @Jourdan-He, thanks for reaching out. Could you provide debug logs for a failed upload and a successful upload? You can get debug logs by adding --debug to your command, and redacting any sensitive information. Thanks!

[github-actions[bot]]

Hello! Reopening this discussion to make it searchable.