Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improvements for xarray provider #1800

Merged
merged 15 commits into from
Sep 30, 2024
Merged

Conversation

barbuz
Copy link
Contributor

@barbuz barbuz commented Aug 27, 2024

Overview

Using the xarray provider to manage Zarr data extensively, I have noticed and fixed a few bugs and added a couple of features that could be useful, so I would like to contribute these changes.

Additional information

Summary of changes:

  • Avoid crashes when the time dimension is not cf-compliant by trying to reload the data with decode_times=False
  • Allow managing data without a time dimension at all
  • The provider was already automatically managing subsets on coordinates sorted in reverse order, now it can do so also when applying a bounding box
  • Fixed bug where some values where not converted to float64 before generating json output
  • Used a named temporary file to generate netcdf output to allow use of netcdf4 engine
  • A couple of minor changes
  • Allow requesting for netcdf or zarr output format regardless of the data format

This last change is the trickiest. I had to patch the APIRequest class to advertise the capability of producing more than one kind of output format for coverage data. At the moment I have hardcoded the two formats supported by the xarray provider, but eventually it would be nice to allow each provider to advertise an arbitrary list of output formats; I'm proposing this change as an initial implementation and a future PR could be able to expand on it and make this available to all providers. Please let me know what you think about this.

Dependency policy (RFC2)

  • I have ensured that this PR meets RFC2 requirements

Updates to public demo

Contributions and licensing

(as per https://github.com/geopython/pygeoapi/blob/master/CONTRIBUTING.md#contributions-and-licensing)

  • I'd like to contribute [feature X|bugfix Y|docs|something else] to pygeoapi. I confirm that my contributions to pygeoapi will be compatible with the pygeoapi license guidelines at the time of contribution
  • I have already previously agreed to the pygeoapi Contributions and Licensing Guidelines

@tomkralidis
Copy link
Member

@barbuz are you able to rebase this PR? I'd rather cover the format handling as part of another issue/discussion. Are you able to reduce this PR to the pure Xarray fixes?

@barbuz
Copy link
Contributor Author

barbuz commented Sep 30, 2024

@barbuz are you able to rebase this PR? I'd rather cover the format handling as part of another issue/discussion. Are you able to reduce this PR to the pure Xarray fixes?

I have rebased this reverting the changes to the format handling. I have opened a separate issue to discuss format handling for coverage providers in general: #1821

@tomkralidis tomkralidis added this to the 0.19.0 milestone Sep 30, 2024
@tomkralidis tomkralidis merged commit d240a82 into geopython:master Sep 30, 2024
4 checks passed
@barbuz barbuz deleted the xarray-improve branch October 3, 2024 00:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants