Crusty

Unofficial WavesEnterprise blockchain crawler. The name is quite unoriginal, I know.

Goal

A replacement for my Python crawling script I've been using for quite a while. More recent versions of WavesEnterprise blockchain node support public gRPC API with a method that streams Blockchain Events. That gives the opportunities to implement services that would subscribe to these events and perform their tasks in event-driven fashion.

And well, it's just fun to write some useful stuff using some tech I don't really know yet. What a great challenge to learn, isn't it?

Tips and tricks

1. If you're using Intellij Rust plugin and have trouble with code completion with include_proto!, try navigating to Experimental Features and enabling org.rust.cargo.evaluate.build.scripts. Worked for me!
2. Assure that you have protoc installed locally. If it is not, install it.

How to launch

RUST_LOG=crusty=trace cargo run config/local.toml

TODOs:

• "Hello world!" compiles and runs (yay!)
• Reads the first command line argument as a path to the config file
• Has a defined Config structure
• Parses TOML config file to a structure
• Enable ENV overriding some config settings (would be handy for passwords)
• Build Rust sources from WE proto-files
• Connect to MongoDB
• Establishes a gRPC connection to the node
• Subscribe to Blockchain Events stream
• Introduce logging
• Drop MongoDB ideas
• Connect to ArangoDB
• Test run crawling of first 10 blocks into ArangoDB
• ... decide what to do next ...
• Pack it all up in a docker image
• ...
• Implement tests
• Create a CI pipeline to launch tests automatically
• ...
• Support REST API
• ...
• Implement an algorithm to calculate leaser payouts
• ...
• Automate payouts
• ...
• API for fancy statistics and other cool stuff
• ...
• Who knows where the road will lead us? Further researches, etc.