STAC specific configurations for Nginx data access

[fmigneault]

You mentioned that you could add some STAC specific settings later on in a different PR. No need to do that now but could you make an issue that outlines some of your ideas so that we don't lose track of them.

Originally posted by @mishaschwartz in #403 (review)

The optional-components/stac-data-proxy allows the platform to share file contents referenced by STAC collections and items externally. Since this content can have variable media-types, sizes, stream strategies, the optimized methods to expose this data efficiently must be explored, such that Nginx configurations provide the best user experience possible.

A few initial ideas:

• STAC employs collections to regroup items and assets. Specific subdirectories will therefore most probably be used to categorize those elements.

  • Cowbird could be used to sync access between those file-directories from nested secure-data-proxy/stac/<collection-id> under Magpie, and the corresponding stac/stac/<collection-id> to access the STAC API through Magpie/Twitcher.
  • A similar concept to Magpie's browse/read used for THREDDS could be defined for STAC-API (browse) vs stac-data-proxy (read).

• Common assets in STAC that could require Nginx optimizations for file-stream transfer

  • GeoTiff
  • COG (cloud-optimized geotiff) (possible sub-tiling filters)
  • PNG/JPEG (mostly for thumbnail)
  • GeoJSON (labels / vectors for datasets)
  • parquet/dataframe (growing use, eg: https://github.com/stac-utils/stac-geoparquet)
  • model references (in exploration) (pytorch .ckpt, ONNX, etc.)
  • others?

• stac-data-proxy could allow file-upload (only download for now via Magpie read permission)

  Given that the file structure to be used under the directory aliased by Nginx could vary depending on necessary optimizations, providing a way to upload STAC assets at the same time as pushing STAC Collection/Item references to STAC-API could require some more advanced logic.