Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

5 overpass download #49

Merged
merged 21 commits into from
Jul 3, 2024
Merged

5 overpass download #49

merged 21 commits into from
Jul 3, 2024

Conversation

annajungbluth
Copy link
Contributor

Overview of changes:

  • added script (downloader_msg_modis_overpass) to download msg timestamps whenever MODIS has an overpass. Note that an equivalent script does not yet exist for GOES
  • changed try/except loop in msg download.py to avoid the program from crashing when no product or file is found. With this, I was able to download data for all of 2010
  • added functionality to add either specific timestamps, or date/time ranges to GOES and MSG download.py. This way, it's easier for the users to query specific data (for instance with MODIS overpasses)
  • added scripts to download MSG data in monthly chuncks. Even with some refactoring, the download is very slow, so it's better to parallelize.

Copy link
Member

@jejjohnson jejjohnson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. The only think I suggest is just to loop through some dates using bash in a single bash script.

Here’s an example script.

#!/bin/bash

SAVE_DIR=...

# Define an array of start and end dates
dates=(
  "2024-01-01 2024-01-31"
  "2024-02-01 2024-02-28"
  "2024-03-01 2024-03-31"
)

# Loop through each pair of dates
for date_pair in "${dates[@]}"; do
  # Split the date_pair into start_date and end_date
  START_DATE=$(echo $date_pair | awk '{print $1}')
  END_DATE=$(echo $date_pair | awk '{print $2}')

  # Print the start and end dates
  echo "Start Date: $START_DATE, End Date: $END_DATE"
  
  # code to process dates
  python rs_tools/_src/data/msg/downloader_msg_modis_overpass.py --save-dir $SAVE_DIR --start-date $START_DATE --end-date $END_DATE

merge when you’re ready! @annajungbluth

@annajungbluth annajungbluth merged commit 17bffb3 into main Jul 3, 2024
0 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants