Burst has a multi-phase language pipeline.
The phases are:
eql-- user-friendly, high-level, declarative languagehydra-- high-performance, low-level, execution languagescala-- code generation platformbytecode-- portable direct execution platform
Behavioral analysis queries are written in EQL which is transpiled to Hydra on the
supervisor-node. The Hydra is parsed and validated on the supervisor-node and then sent
to the active worker-node instances. The Hydra is reparsed, validated, on the
worker-node
and then goes through a code generation phase, resulting in Scala code, which results in
byte-code closures which perform direct scan traversal and analytic processing
of the encoded binary Brio object-tree blobs.
EQL is the front-end language for Burst. Though it is
possible to Generally
speaking this is the only public external facing language API for Burst. It is
designed to support effective and efficient authoring of
Burst analyses. EQL knows how to translate (transpile) these human friendly
declarative source statements into its own internal
single-pass-scan model and thus to the fully imperative
Hydra language.
Hydra is the back-end language for Burst. Hydra
(and its underlying semantic model Felt) is where
the single-pass-scan, parallel, distributed, partitioned,
scatter-gather model of Burst is activated. The EQL statements become
Hydra types and behavior that directs the system
as it collects and processes data on the worker node and sends it back to the
supervisor node.
Hydra generates Scala source language defining types and behavior that directly define all the low level data structures and algorithms that know how to scan the Brio blob and produce result sets for the given analysis.
The generated Scala source language is then compiled into byte-code. This is the most expensive part of the language processing pipeline other than the scan itself of course.
Since the Scala compilation is the most expensive part of the pipeline, and in order to benefit from the full array of JIT compiling (warming) in the Java world. It is important to understand the