Skip to content

Commit

Permalink
Merge pull request #2891 from nasa/jk/CUMULUS-1/release-10.1.2
Browse files Browse the repository at this point in the history
  • Loading branch information
Jkovarik authored Mar 11, 2022
2 parents c74f95a + 58796c2 commit 3b59753
Show file tree
Hide file tree
Showing 70 changed files with 335 additions and 380 deletions.
52 changes: 3 additions & 49 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,54 +6,7 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).

## Unreleased

## [v10.1.1] 2022-03-04

### Migration steps

- Due to a bug in the PUT `/rules/<name>` endpoint, the rule records in PostgreSQL may be
out of sync with records in DynamoDB. In order to bring the records into sync, re-run the
[previously deployed `data-migration1` Lambda](https://nasa.github.io/cumulus/docs/upgrade-notes/upgrade-rds#3-deploy-and-run-data-migration1) with a payload of
`{"forceRulesMigration": true}`:

```shell
aws lambda invoke --function-name $PREFIX-data-migration1 \
--payload $(echo '{"forceRulesMigration": true}' | base64) $OUTFILE
```

### Added

- **CUMULUS-2846**
- Added `@cumulus/db/translate/rule.translateApiRuleToPostgresRuleRaw` to translate API rule to PostgreSQL rules and
**keep undefined fields**

### Changed

- **CUMULUS-NONE**
- Adds logging to ecs/async-operation Docker conatiner that launches async
tasks on ECS. Sets default `async_operation_image_version` to 39.
- **CUMULUS-2845**
- Updated rules model to decouple `createRuleTrigger` from `create`.
- Updated rules POST endpoint to call `rulesModel.createRuleTrigger` directly to create rule trigger.
- Updated rules PUT endpoints to call `rulesModel.createRuleTrigger` if update fails and reversion needs to occur.
- **CUMULUS-2846**
- Updated version of `localstack/localstack` used in local unit testing to `0.11.5`

### Fixed

- Upgraded lodash to version 4.17.21 to fix vulnerability
- **CUMULUS-2845**
- Fixed bug in POST `/rules` endpoint causing rule records to be created
inconsistently in DynamoDB and PostgreSQL
- **CUMULUS-2846**
- Fixed logic for `PUT /rules/<name>` endpoint causing rules to be saved
inconsistently between DynamoDB and PostgreSQL
- **CUMULUS-2854**
- Fixed queue granules behavior where the task was not accounting for granules that
*already* had createdAt set. Workflows downstream in this scenario should no longer
fail to write their granules due to order-of-db-writes constraints in the database
update logic.

## [v10.1.0] 2022-02-23
## [v10.1.2] 2022-03-11

### Added

Expand Down Expand Up @@ -5400,7 +5353,8 @@ Note: There was an issue publishing 1.12.0. Upgrade to 1.12.1.

## [v1.0.0] - 2018-02-23

[unreleased]: https://github.com/nasa/cumulus/compare/v10.1.1...HEAD
[unreleased]: https://github.com/nasa/cumulus/compare/v10.1.2...HEAD
[v10.1.2]: https://github.com/nasa/cumulus/compare/v10.1.1...v10.1.2
[v10.1.1]: https://github.com/nasa/cumulus/compare/v10.1.0...v10.1.1
[v10.1.0]: https://github.com/nasa/cumulus/compare/v10.0.1...v10.1.0
[v10.0.1]: https://github.com/nasa/cumulus/compare/v10.0.0...v10.0.1
Expand Down
2 changes: 1 addition & 1 deletion example/lambdas/asyncOperations/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@cumulus/test-async-operations",
"version": "10.1.1",
"version": "10.1.2",
"description": "AsyncOperations Test Lambda",
"main": "index.js",
"private": true,
Expand Down
4 changes: 2 additions & 2 deletions example/lambdas/ftpPopulateTestLambda/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@cumulus/ftp-populate-test-lambda",
"version": "10.1.0",
"version": "10.1.2",
"description": "FTP Population Utility Lambda",
"main": "index.js",
"private": true,
Expand All @@ -19,8 +19,8 @@
"access": "private"
},
"dependencies": {
"@cumulus/api-client": "10.1.0",
"@cumulus/api": "10.1.0",
"@cumulus/api-client": "10.1.0",
"@cumulus/aws-client": "10.1.0",
"@cumulus/common": "10.1.0",
"@cumulus/integration-tests": "10.1.0",
Expand Down
2 changes: 1 addition & 1 deletion example/lambdas/python-processing/package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "@cumulus/python-process-activity",
"private": true,
"version": "10.1.1",
"version": "10.1.2",
"description": "Python reference activity",
"homepage": "https://github.com/nasa/cumulus/tree/master/example/lambdas/python-reference-activity",
"repository": {
Expand Down
2 changes: 1 addition & 1 deletion example/lambdas/python-reference-activity/package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "@cumulus/python-reference-activity",
"private": true,
"version": "10.1.1",
"version": "10.1.2",
"description": "Python reference activity",
"homepage": "https://github.com/nasa/cumulus/tree/master/example/lambdas/python-reference-activity",
"repository": {
Expand Down
2 changes: 1 addition & 1 deletion example/lambdas/python-reference-task/package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "@cumulus/python-reference-task",
"private": true,
"version": "10.1.1",
"version": "10.1.2",
"description": "Python reference task",
"main": "index.js",
"homepage": "https://github.com/nasa/cumulus/tree/master/example/lambdas/python-reference-task",
Expand Down
2 changes: 1 addition & 1 deletion example/lambdas/s3AccessTest/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@cumulus/test-s3-access",
"version": "10.1.1",
"version": "10.1.2",
"description": "S3 Access Test Lambda",
"main": "index.js",
"private": true,
Expand Down
2 changes: 1 addition & 1 deletion example/lambdas/snsS3Test/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@cumulus/test-sns-s3",
"version": "10.1.1",
"version": "10.1.2",
"description": "SNS to S3 Test Lambda",
"main": "index.js",
"private": true,
Expand Down
2 changes: 1 addition & 1 deletion example/lambdas/versionUpTest/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@cumulus/test-version-up",
"version": "10.1.1",
"version": "10.1.2",
"description": "Version Up Test Lambda",
"main": "index.js",
"private": true,
Expand Down
52 changes: 26 additions & 26 deletions example/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@cumulus/cumulus-integration-tests",
"version": "10.1.1",
"version": "10.1.2",
"description": "Cumulus Integration Test Deployment",
"private": true,
"main": "index.js",
Expand Down Expand Up @@ -45,32 +45,32 @@
]
},
"dependencies": {
"@cumulus/api": "10.1.1",
"@cumulus/api-client": "10.1.1",
"@cumulus/async-operations": "10.1.1",
"@cumulus/aws-client": "10.1.1",
"@cumulus/checksum": "10.1.1",
"@cumulus/cmr-client": "10.1.1",
"@cumulus/cmrjs": "10.1.1",
"@cumulus/common": "10.1.1",
"@cumulus/discover-granules": "10.1.1",
"@cumulus/discover-pdrs": "10.1.1",
"@cumulus/files-to-granules": "10.1.1",
"@cumulus/hello-world": "10.1.1",
"@cumulus/ingest": "10.1.1",
"@cumulus/integration-tests": "10.1.1",
"@cumulus/message": "10.1.1",
"@cumulus/move-granules": "10.1.1",
"@cumulus/parse-pdr": "10.1.1",
"@cumulus/pdr-status-check": "10.1.1",
"@cumulus/post-to-cmr": "10.1.1",
"@cumulus/queue-granules": "10.1.1",
"@cumulus/queue-pdrs": "10.1.1",
"@cumulus/sf-sqs-report": "10.1.1",
"@cumulus/sync-granule": "10.1.1",
"@cumulus/test-processing": "10.1.1"
"@cumulus/api": "10.1.2",
"@cumulus/api-client": "10.1.2",
"@cumulus/async-operations": "10.1.2",
"@cumulus/aws-client": "10.1.2",
"@cumulus/checksum": "10.1.2",
"@cumulus/cmr-client": "10.1.2",
"@cumulus/cmrjs": "10.1.2",
"@cumulus/common": "10.1.2",
"@cumulus/discover-granules": "10.1.2",
"@cumulus/discover-pdrs": "10.1.2",
"@cumulus/files-to-granules": "10.1.2",
"@cumulus/hello-world": "10.1.2",
"@cumulus/ingest": "10.1.2",
"@cumulus/integration-tests": "10.1.2",
"@cumulus/message": "10.1.2",
"@cumulus/move-granules": "10.1.2",
"@cumulus/parse-pdr": "10.1.2",
"@cumulus/pdr-status-check": "10.1.2",
"@cumulus/post-to-cmr": "10.1.2",
"@cumulus/queue-granules": "10.1.2",
"@cumulus/queue-pdrs": "10.1.2",
"@cumulus/sf-sqs-report": "10.1.2",
"@cumulus/sync-granule": "10.1.2",
"@cumulus/test-processing": "10.1.2"
},
"devDependencies": {
"@cumulus/test-data": "10.1.1"
"@cumulus/test-data": "10.1.2"
}
}
6 changes: 3 additions & 3 deletions example/scripts/generate_ingest/package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "@cumulus/generate_ingest",
"private": true,
"version": "10.1.1",
"version": "10.1.2",
"description": "Script to generate test data for scaled ingest",
"keywords": [
"GIBS",
Expand All @@ -22,8 +22,8 @@
"directory": "packages/types"
},
"dependencies": {
"@cumulus/aws-client": "10.1.1",
"@cumulus/common": "10.1.1"
"@cumulus/aws-client": "10.1.2",
"@cumulus/common": "10.1.2"
},
"author": "Cumulus Authors",
"license": "Apache-2.0"
Expand Down
18 changes: 9 additions & 9 deletions lambdas/data-migration1/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@cumulus/data-migration1",
"version": "10.1.1",
"version": "10.1.2",
"description": "A Lambda function used for doing data migrations",
"license": "Apache-2.0",
"engines": {
Expand All @@ -25,18 +25,18 @@
"timeout": "15m"
},
"dependencies": {
"@cumulus/api": "10.1.1",
"@cumulus/aws-client": "10.1.1",
"@cumulus/common": "10.1.1",
"@cumulus/db": "10.1.1",
"@cumulus/errors": "10.1.1",
"@cumulus/logger": "10.1.1",
"@cumulus/types": "10.1.1",
"@cumulus/api": "10.1.2",
"@cumulus/aws-client": "10.1.2",
"@cumulus/common": "10.1.2",
"@cumulus/db": "10.1.2",
"@cumulus/errors": "10.1.2",
"@cumulus/logger": "10.1.2",
"@cumulus/types": "10.1.2",
"knex": "0.95.15",
"lodash": "^4.17.21",
"pg": "^8.3.0"
},
"devDependencies": {
"@cumulus/test-data": "10.1.1"
"@cumulus/test-data": "10.1.2"
}
}
20 changes: 10 additions & 10 deletions lambdas/data-migration2/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@cumulus/data-migration2",
"version": "10.1.1",
"version": "10.1.2",
"description": "A Lambda function used for doing data migrations intended to be executed after data-migration1.",
"license": "Apache-2.0",
"engines": {
Expand All @@ -25,21 +25,21 @@
"timeout": "15m"
},
"dependencies": {
"@cumulus/api": "10.1.1",
"@cumulus/aws-client": "10.1.1",
"@cumulus/common": "10.1.1",
"@cumulus/db": "10.1.1",
"@cumulus/errors": "10.1.1",
"@cumulus/logger": "10.1.1",
"@cumulus/message": "10.1.1",
"@cumulus/types": "10.1.1",
"@cumulus/api": "10.1.2",
"@cumulus/aws-client": "10.1.2",
"@cumulus/common": "10.1.2",
"@cumulus/db": "10.1.2",
"@cumulus/errors": "10.1.2",
"@cumulus/logger": "10.1.2",
"@cumulus/message": "10.1.2",
"@cumulus/types": "10.1.2",
"JSONStream": "1.3.5",
"knex": "0.95.15",
"lodash": "^4.17.21",
"p-map": "^4.0.0",
"pg": "^8.3.0"
},
"devDependencies": {
"@cumulus/test-data": "10.1.1"
"@cumulus/test-data": "10.1.2"
}
}
4 changes: 2 additions & 2 deletions lambdas/db-migration/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@cumulus/db-migration-lambda",
"version": "10.1.1",
"version": "10.1.2",
"description": "A Lambda function used for deploying DB migrations",
"license": "Apache-2.0",
"engines": {
Expand All @@ -20,7 +20,7 @@
"tsc:listEmittedFiles": "../../node_modules/.bin/tsc --listEmittedFiles"
},
"dependencies": {
"@cumulus/db": "10.1.1",
"@cumulus/db": "10.1.2",
"knex": "0.95.15",
"pg": "^8.3.0"
}
Expand Down
6 changes: 3 additions & 3 deletions lambdas/db-provision-user-database/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@cumulus/db-provision-user-database-lambda",
"version": "10.1.1",
"version": "10.1.2",
"description": "A Lambda function used for provisioning user databases",
"engines": {
"node": ">=12.18.0"
Expand All @@ -24,8 +24,8 @@
"timeout": "2m"
},
"dependencies": {
"@cumulus/common": "10.1.1",
"@cumulus/db": "10.1.1",
"@cumulus/common": "10.1.2",
"@cumulus/db": "10.1.2",
"knex": "0.95.15",
"pg": "^8.3.0"
},
Expand Down
12 changes: 6 additions & 6 deletions lambdas/postgres-migration-async-operation/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@cumulus/postgres-migration-async-operation",
"version": "10.1.1",
"version": "10.1.2",
"description": "A Lambda function used to start an ECS task to run data-migrations2 lambda",
"license": "Apache-2.0",
"engines": {
Expand Down Expand Up @@ -28,12 +28,12 @@
"timeout": "15m"
},
"dependencies": {
"@cumulus/api": "10.1.1",
"@cumulus/async-operations": "10.1.1",
"@cumulus/logger": "10.1.1",
"@cumulus/types": "10.1.1"
"@cumulus/api": "10.1.2",
"@cumulus/async-operations": "10.1.2",
"@cumulus/logger": "10.1.2",
"@cumulus/types": "10.1.2"
},
"devDependencies": {
"@cumulus/test-data": "10.1.1"
"@cumulus/test-data": "10.1.2"
}
}
14 changes: 7 additions & 7 deletions lambdas/postgres-migration-count-tool/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@cumulus/postgres-reconcile",
"version": "10.1.1",
"version": "10.1.2",
"description": "A Lambda function used for generating counts between Dynamo/ES and Postgres",
"license": "Apache-2.0",
"engines": {
Expand All @@ -21,12 +21,12 @@
"tsc:listEmittedFiles": "../../node_modules/.bin/tsc --listEmittedFiles"
},
"dependencies": {
"@cumulus/api": "10.1.1",
"@cumulus/api-client": "10.1.1",
"@cumulus/common": "10.1.1",
"@cumulus/db": "10.1.1",
"@cumulus/message": "10.1.1",
"@cumulus/types": "10.1.1",
"@cumulus/api": "10.1.2",
"@cumulus/api-client": "10.1.2",
"@cumulus/common": "10.1.2",
"@cumulus/db": "10.1.2",
"@cumulus/message": "10.1.2",
"@cumulus/types": "10.1.2",
"knex": "0.95.15",
"p-map": "^4.0.0",
"pg": "^8.3.0"
Expand Down
Loading

0 comments on commit 3b59753

Please sign in to comment.