Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stream GPM brightness temperature and precip with earthaccess #42

Open
wants to merge 30 commits into
base: main
Choose a base branch
from

Conversation

zmoon
Copy link
Collaborator

@zmoon zmoon commented Dec 19, 2024

Closes #38

zmoon added 19 commits December 6, 2024 15:42
couldn't link to the howto/authenticate page, I guess
since it's using mkdocs not sphinx
it's HDF5 but not nc
all the variables are in this group
though the root does have some ds attrs like DOI

Example read result with no modifications:

<xarray.Dataset>
Dimensions:                         (time: 1, lon: 3600, lat: 1800, nv: 2,
                                     lonv: 2, latv: 2)
Coordinates:
  * time                            (time) object 2023-08-01 00:00:00
  * lon                             (lon) float32 -179.9 -179.9 ... 179.8 179.9
  * lat                             (lat) float32 -89.95 -89.85 ... 89.85 89.95
Dimensions without coordinates: nv, lonv, latv
Data variables:
    time_bnds                       (time, nv) object ...
    lon_bnds                        (lon, lonv) float32 ...
    lat_bnds                        (lat, latv) float32 ...
    precipitation                   (time, lon, lat) float32 ...
    randomError                     (time, lon, lat) float32 ...
    probabilityLiquidPrecipitation  (time, lon, lat) float32 ...
    precipitationQualityIndex       (time, lon, lat) float32 ...
Attributes:
    GridHeader:  BinMethod=ARITHMETIC_MEAN;\nRegistration=CENTER;\nLatitudeRe...
(Pdb) ds.lon_bnds
<xarray.DataArray 'lon_bnds' (lon: 3600, lonv: 2)>
[7200 values with dtype=float32]
Coordinates:
  * lon      (lon) float32 -179.9 -179.9 -179.8 -179.6 ... 179.8 179.8 179.9
Dimensions without coordinates: lonv
Attributes:
    DimensionNames:  lon,lonv
    Units:           degrees_east
    units:           degrees_east
In a quick comparison involving 97 IMERG files, ~ 6 min -> 4 min
tried to do period with two positional args by accident,
this prevents that

can make the arg positional-only when changing python_requires
@zmoon zmoon requested a review from Copilot December 19, 2024 22:41

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 6 out of 7 changed files in this pull request and generated no comments.

Files not reviewed (1)
  • docs/api.rst: Language not supported
Comments suppressed due to low confidence (3)

tams/data.py:414

  • The error message could be more specific. Suggestion: 'time_or_range must be a single time or a tuple of two times (e.g., '2024-06-01 02:30' or ('2024-06-01 02:30', '2024-06-01 03:00')) in a format accepted by pandas.to_datetime.'
`time_or_range` must be a single time or a tuple of two times in a format accepted by pandas.to_datetime

tams/data.py:421

  • The TODO comment should be addressed or removed to avoid leaving unfinished tasks in the code.
time_or_range: Any | tuple[Any, Any],  # TODO: positional-only

tams/data.py:514

  • The TODO comment should be addressed or removed to avoid leaving unfinished tasks in the code.
time_or_range: Any | tuple[Any, Any],  # TODO: positional-only
@zmoon
Copy link
Collaborator Author

zmoon commented Dec 20, 2024

Need to add Earthdata credentials as secrets for the nb to work on RTD. Also check .authenticated on the return from login, since the current error is a bit confusing

@zmoon
Copy link
Collaborator Author

zmoon commented Dec 20, 2024

New raise working on RTD:

image

@zmoon
Copy link
Collaborator Author

zmoon commented Dec 20, 2024

Looks like PR builds don't get the secrets, which is why the nb is still failing in RTD even though I added the env vars. We don't want this to fail the docs build every PR, so I guess a compromise is to allow errors for this nb only.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Stream input data with earthaccess
1 participant