Convert the ECMWF model level products to pressure level products #31
Conversation
|
@scottstanie @mirzaaes can you please review this PR? We would like to have a quick turn around on this and I believe this PR should be also beneficial for your projects. Thank you. |
There was a problem hiding this comment.
Thank you @xhuang-jpl, this is a useful PR specially once we start using HRES data. It looks good to me, however it would be nice if you could add docstring and parameter description for those functions that they don't have
| count = 0 | ||
| profile_count = 0 | ||
| interpolated_values=[] | ||
| for point in range(no_grid_points): |
There was a problem hiding this comment.
This variable is not used, and will simply replace it with _
| def get_input_variable_list(fin): | ||
| ds = xr.open_dataset(fin) | ||
| var_list = list(ds.keys()) | ||
| ds = None | ||
| var_list_unique = list(set(var_list)) | ||
| if 'lnsp' not in var_list_unique: | ||
| print("Error - lnsp variable missing from input file -exiting") | ||
| sys.exit() | ||
| if len(var_list_unique) < 2: | ||
| print("Error - Data variable missing from input file -exiting") | ||
| sys.exit() | ||
| return var_list_unique |
There was a problem hiding this comment.
A couple general comments which carry throughout the module-
- For these general functions, let's raise a meaningful exception rather than call
sys.exit - you appear to sometimes use 2 spaces for indentation rather than 4
- a docstring describing the inputs/outputs would be helpful. for this function, it's hard to guess just from looking at these 10 lines what
finshould be, or where i could find a file with'lnsp'as anxarray-readable variable - As we mentioned offline, let's double check- is xarray needed for this? or is it just used to read in the inputs (which are GRIB files readable by
pygrib?) and write the outputs
For this specific function
- this function just open a file and gets the variables, but then we reopen and use the file right after
- it we're doing this anyway, we should use the context manager version of
with xr.open_dataset(fin):.... It's only the Python-gdal bindings that require you to dods = Noneto close a dataset - I can't tell what
var_list_uniqueis supposed to contain that is a problem if the length is < 2
There was a problem hiding this comment.
Thank you @scottstanie, I think we might need the netcdf reader here because the NISAR PCM has already the interface to generate the netcdf to feed the ISCE3, let me find a cheaper way to do this, maybe only the netcdf reader instead of the xarray.
|
I am a little concerned about how to deal with those NaNs that are out range of the pressure, interestingly, the downloaded pressure level products do not have this issue. I am thinking two options, |
|
Since there is already GRIB test data in the repo, it should be quick to add a test of the main conversion function |
@scottstanie @mirzaees @vbrancat
To make the pyAPS accept the ECMWF model level products, this PR is to convert the model level ECMWF products in NetCDF format to the pressure level products. Both the theory and partial codes for the conversion can be found in ECMWF website. This PR is tested comparing with the ECMWF pressure level products. Some tests results are shown as,
1. Temperature
2. Specific Humidity
3. Geopotential