This app reads from Firehose Kinesis streams, decodes the records using the appropriate Avro schema based on the stream name, and returns the resulting records as either JSON or CSV (base64 encoded). This app is responsible for decoding records immediately before ingest into the BIC.
v1.0.1
Install all Node dependencies via NPM
nvm use
npm install
Use the sam cli to run the lambda on arbitrary firehose events. To process a firehose event containing 3 CircTrans records and print out the result:
sam local invoke --profile nypl-digital-dev -t sam.qa.yml -e sample/firehose-CircTrans-3-records-encoded.json
This repo uses the "PRs Target Main, Merge to Deployment Branches" git workflow:
- Cut PRs from
main - Merge
main>qa - Merge
main>production
This app is deployed via Travis-CI using terraform. Code in qa is pushed to AvroToJsonTransformer-qa. Code in production is pushed to AvroToJsonTransformer-production.
To run all tests found in ./test/:
npm run test
To run a specific test for the given filename:
npm run test [filename].test.js
This repo uses c8 to compute test coverage (because Istanbul doesn't appear to support ESM at writing). Coverage reports are included at the end of npm test. For a detailed line-by-line breakdown, view the HTML report:
npm run coverage-report
open coverage/index.html
This codebase uses Standard JS as the JavaScript linter.
To check for linting errors:
npm run lint