quickview-streaming

A cookiecutter-based project to stream data from liquidity pool¹.

Configuration

• Clone current repository:

git clone https://github.com/e183b796621afbf902067460/quickview-streaming.git

• Get into the project folder:

cd quickview-streaming/

• Set WSS_NODE_PROVIDER environment variable in docker-compose.yaml.

Quickstart

• Create venv and activate it:

python3 -m venv venv && source venv/bin/activate

Dependency Management

Project's dependencies are managed by the poetry². Moving to poetry has helped to have pyproject.toml as a single configuration file for the whole project as well.

• Install poetry:

pip3 install poetry  # 1.7.1

• And install project's dependencies:

poetry install --no-root

Add New Dependency

• If a new dependency have to be added, then it can be done this way, for example to add fastapi:

poetry add fastapi

• And lock it:

poetry lock --no-update

How To Use Pre-Commit Hooks

pre-commit³ is used to run pre-commit hooks. All configurations are in the .pre-commit-config.yaml. To run commands below all dependencies have to be installed.

• To initialize hook:

pre-commit install

• To check all files, not only files in the commit:

pre-commit run --all-files

Configuration

Kafka environment variables

• BOOTSTRAP_SERVERS: Kafka application bootstrap servers.
• TOPIC_NAME: Kafka application topic name.

Node environment variables

• WSS_NODE_PROVIDER: WebSocket node provider.

Deploy

• Run all in docker, perhaps using sudo:

docker-compose up -d --build --force-recreate

• Check KafkaUI⁴ streaming data.

• Stop all services:

docker-compose down -v

Based on quickview-template⁵.

Footnotes

1. https://polygonscan.com/address/0xAE81FAc689A1b4b1e06e7ef4a2ab4CD8aC0A087D. ↩

2. https://python-poetry.org/. ↩

3. https://pre-commit.com/. ↩

4. http://0.0.0.0:8080/. ↩

5. https://github.com/e183b796621afbf902067460/quickview-template. ↩