Chainlink Oracles Data Bridge to Secret Network

General

This mono repo contains the source of services and contracts development projects for relaying Chainlink oracle contracts' data towards Secret Network contracts.

The overall solution and thoughts are reported in this project wiki.

The project was initiated and developped in the context of the Chainlink Spring 2021 Hackathon. The initial proof-of-concept submission made on 2021-04-11 is available in devpost.com here.

Some of the key tools / tech this project benefits from:

• NestJS Best TypeScript app server development option
• Ethers.js Ethereum network client SDK in TypeScript
• Secret Network toolkits, reference docs and sample apps
• CosmWasm excellent Cosmos toolkits and reference documentation
• RustUp full featured Rust toolchain manager, with Cargo & rustc
• Chainlink tools, vision and their off-chain / on-chain integration design
• Kafka as the distributed event streaming platform among the main services

Status

This dev project has been put aside, considering mostly that the BandProtocol is a good & existing option for Secret contracts to integrate data/price feeds.

The interest of this code repository is mostly on the ETH & SN SDKs integration, and moreover the solution for independant MS operators to collaborate through Kakfka: to distribute the tasks/workloads and reach a consensus to apply changes on chain.

Tech Requirements

• Yarn to build & run the modules
• Lerna in order to develop against local versions of packages
• Kafka for the services to interact
• Docker for running the services locally containerized, and fully optimizing the Secret contracts
• Cargo and corresponding Rust toolkit for the Secret oracle contract development
• GNU Make to benefit from build scripts

Each module should define its own dependencies and provide minimum guidance. Still there are many different frameworks and technics used in order the full end-to-end solution.

Play

Setup

lerna bootstrap

Installs packages for all workspaces. Except for the Secret oracle contracts' dev projects.

Build

lerna run build

Runs the setup step for all modules. Typically this step just compiles TypeScript, but may involve other tasks.

Test

In order to test modules locally, you may need to set environment variables for the given API.

lerna run test

Docker

To build a Docker container for a specific $module, use the following example:

make docker module=moduleName

The naming convention for Docker containers will be $moduleName-module.

Then run it with:

docker run -p 8080:8080 -e API_KEY='YOUR_API_KEY' -it moduleName-module:latest

Serverless

Create the zip:

make zip module=moduleName

The zip will be created as ./$moduleName/dist/$moduleName-module.zip.

Install to AWS Lambda

• In Lambda Functions, create function
• On the Create function page:
  • Give the function a name
  • Use Node.js 12.x for the runtime
  • Choose an existing role or create a new one
  • Click Create Function
• Under Function code, select "Upload a .zip file" from the Code entry type drop-down
• Click Upload and select the $moduleName-module.zip file
• Handler:
  • index.handler (same as index.awsHandlerREST) for REST API Gateways (AWS Lambda default)
  • index.awsHandlerREST for REST API Gateways
  • index.awsHandlerHTTP for HTTP API Gateways
• Add the environment variable (repeat for all environment variables):
  • Key: API_KEY
  • Value: Your_API_key
• Save

By default, Lambda functions time out after 3 seconds. You may want to change that to 60s in case an API takes longer than expected to respond.

To Set Up an API Gateway (HTTP API)

If using a HTTP API Gateway, Lambda's built-in Test will fail, but you will be able to externally call the function successfully.

• Click Add Trigger
• Select API Gateway in Trigger configuration
• Under API, click Create an API
• Choose HTTP API
• Select the security for the API
• Click Add

To Set Up an API Gateway (REST API)

If using a REST API Gateway, you will need to disable the Lambda proxy integration for Lambda-based adapter to function.

• Click Add Trigger
• Select API Gateway in Trigger configuration
• Under API, click Create an API
• Choose REST API
• Select the security for the API
• Click Add
• Click the API Gateway trigger
• Click the name of the trigger (this is a link, a new window opens)
• Click Integration Request
• Uncheck Use Lamba Proxy integration
• Click OK on the two dialogs
• Click the Mapping Templates dropdown
• Check "When there are no templates defined (recommended)"
• Add new Content-Type application/json
• Use Mapping Template:

#set($input.path("$").queryStringParameters = $input.params().querystring)
$input.json('$')

• Click Save
• Return to your function
• Remove the API Gateway and Save
• Click Add Trigger and use the same API Gateway
• Select the deployment stage and security
• Click Add

Install to GCP

• In Functions, create a new function, choose to ZIP upload
• Select Node.js 12 for the Runtime
• Click Browse and select the $moduleName-module.zip file
• Select a Storage Bucket to keep the zip in
• Function to execute: gcpHandler
• Click More, Add variable (repeat for all environment variables)
  • NAME: API_KEY
  • VALUE: Your_API_key