HLS LPDAAC

Requirements

• uv (installs Python, the virtualenv and every dev tool)
• AWS CLI
• AWS IAM role with sufficient permissions for creating, destroying and modifying relevant stack resources

uv reads the Python version from .python-version and the dependencies from pyproject.toml/uv.lock, so no other Python tooling needs to be installed. The CDK CLI is a Node package; make installs Node and the CLI into .venv, so Node does not need to be installed globally either.

Environment Settings

# AWS Short-term Access Key

export AWS_DEFAULT_REGION=us-west-2
export AWS_ACCESS_KEY_ID=<id>
export AWS_SECRET_ACCESS_KEY=<key>
export AWS_SESSION_TOKEN=<token>

# Stack variables

export HLS_LPDAAC_STACK=<stack name>
export HLS_LPDAAC_BUCKET_NAME=<source bucket name>
export HLS_LPDAAC_QUEUE_ARN=<destination queue ARN>
# Required ONLY in PROD for FORWARD processing (otherwise, a dummy queue is created)
export HLS_LPDAAC_MANAGED_POLICY_NAME=mcp-tenantOperator-APIG

# Optional notification delivery controls (defaults shown)
export HLS_LPDAAC_PAUSED=false
export HLS_LPDAAC_MAX_CONCURRENCY=5
export HLS_LPDAAC_BATCH_SIZE=10

Deployment

Deployments run from GitHub Actions, off the main branch only.

Target	Trigger	GitHub environments
Dev	Merge to main	dev-forward, dev-historical
Prod	Publish a release	prod-forward, prod-historical

Each target deploys both the forward and historical stacks, and only after the unit and integration tests pass. Stack configuration comes from the variables and secrets set on the GitHub environment being deployed to.

A single stack can also be deployed by running the Deploy workflow manually, choosing an environment and its matching target, or from a shell with the CDK commands below.

Pausing notifications during an LPDAAC maintenance window

S3 object-created notifications are delivered to a queue owned by this stack, which the forwarder Lambda consumes through an SQS event source mapping. While the mapping is disabled, notifications accumulate in that queue (14 day retention) and are forwarded once it is re-enabled.

The HLS_LPDAAC_PAUSED GitHub environment variable is the single source of truth for whether a stack is paused. Every deploy, manual or automatic, reads it, so a merge to main or a published release during a maintenance window keeps the stack paused.

To pause, set the variable on the affected environment and redeploy:

gh variable set HLS_LPDAAC_PAUSED --env prod-forward --body true
gh workflow run deploy.yml -f environment=prod-forward -f target=forward

To resume, set it back to false (or delete it) and redeploy:

gh variable set HLS_LPDAAC_PAUSED --env prod-forward --body false
gh workflow run deploy.yml -f environment=prod-forward -f target=forward

Do not disable the event source mapping with the AWS CLI or console: the next deploy reconciles it back to the value declared by CDK.

HLS_LPDAAC_MAX_CONCURRENCY caps concurrent forwarder invocations, smoothing bursts of notifications sent to LPDAAC, and is changed the same way.

CDK Commands

In the make commands shown below, <APP> must be one of the following:

• forward
• forward-it (integration test stack)
• historical
• historical-it (integration test stack)

Synth

Display generated cloud formation template that will be used to deploy.

make synth-<APP>

Diff

Display a diff of the current deployment and any changes created.

make diff-<APP>

Deploy

Deploy current version of stack:

make deploy-<APP>

Destroy

Destroy current version of stack:

make destroy-<APP>

Development

Create the virtualenv in .venv and install the pre-commit hooks:

make install

The hooks configured in .pre-commit-config.yaml then run on every commit. To run them over the whole tree:

uv run pre-commit run --all-files

Run any command inside the virtualenv with uv run <command>, or activate it the usual way:

source .venv/bin/activate

uv keeps .venv in sync with uv.lock on every uv run, so there is no explicit "recreate the environment" step. After changing dependencies in pyproject.toml, refresh the lock file and commit it:

uv lock

Lint, format and type checks

ruff handles both linting and formatting, and mypy handles type checking. Their configuration lives in pyproject.toml.

make lint       # ruff format --check + ruff check
make format     # ruff check --fix + ruff format
make typecheck  # mypy

Tests

To run unit tests:

make unit-tests

To run lint, type checks and unit tests together, as CI does:

make check

To run integration tests for forward processing:

make deploy-forward-it
make forward-integration-tests
make destroy-forward-it

To run integration tests for historical processing:

make deploy-historical-it
make historical-integration-tests
make destroy-historical-it