Add support for changing the UNLOAD S3 file suffix

[dschiavu]

Hi,

It would be great if the UNLOAD command's S3 filename suffix could be made configurable. Currently it's hardcoded as 0000_part_00 in the Writer class:

redshift-fake-driver/src/main/scala/jp/ne/opt/redshiftfake/write/Writer.scala

Line 33 in 96dfe08

val resultKey = s"${unloadCommand.destination.prefix}0000_part_00"

Thanks,
Danijel

[ocadaruma]

Thanks for raising an issue.

Is changing S3 filename suffix possible in AWS Redshift? (sorry, I'm no longer using AWS in my work now so I don't have ideas about this)

If so, it sounds make sense but otherwise, could you elaborate the motivation a bit?

[dschiavu]

Thanks for getting back to me.

Unfortunately it's not possible to tell Redshift to use an alternative S3 UNLOAD file suffix, see https://docs.aws.amazon.com/redshift/latest/dg/r_UNLOAD.html - it always adds the 000 to the file extension, and this suffix is hardcoded.

See also https://stackoverflow.com/questions/39255954/redshift-unloads-file-name

Hence, to make redshift-fake mimic the official AWS Redshift behaviour as closely as possible, I'd suggest to change 0000_part_00 (the current hardcoded suffix) to 000. Or, even better, allow specifying a custom suffix (but still using the 000 as the sensible default).

[dschiavu]

FYI, I'm working on this feature now, planning to raise a Pull Request soon.

[dschiavu]

Done in #75