Skip to content

Latest commit

 

History

History
82 lines (59 loc) · 3.78 KB

information-model.md

File metadata and controls

82 lines (59 loc) · 3.78 KB

Information Model

This section specifies what a valid catalog, workflow, and flow conditions look like.

Table of Contents generated with DocToc

Workflow

  • a single Clojure map which is EDN serializable/deserializable
  • all elements in the map are keywords
  • all elements in the map must correspond to an :onyx/name entry in the catalog
  • the outer-most keys of the map must have catalog entries of :onyx/type that map to :input
  • only innermost values of the map may have catalog entries of :onyx/type that map to :output

Catalog

  • a single Clojure vector which is EDN serializable/deserializable
  • all elements in the vector must be Clojure maps

All maps in the vector must have these keys

key name type choices default
:onyx/name keyword any
:onyx/type keyword :input, :output, :function
:onyx/consumption keyword :sequential, :concurrent
:onyx/batch-size integer >= 0

All maps may optionally have these keys

key name type choices default
:onyx/ident keyword any
:onyx/batch-timeout integer >= 0 1000
:onyx/max-peers integer > 0

Maps with :onyx/type set to :input or :output must have these keys

key name type choices
:onyx/medium keyword any

Maps with :onyx/type set to :function must have these keys

key name type choices
:onyx/fn keyword any

Maps with :onyx/type set to :function may optionally have these keys

key name type choices
:onyx/group-by-key keyword any
:onyx/group-by-fn keyword any

Flow Conditions

  • a single Clojure vector which is EDN serializable/deserializable
  • all elements in the vector must be Clojure maps
key name type optional? default
:flow/from keyword no
:flow/to :all, :none or [keyword] no
:flow/predicate keyword or [keyword] no
:flow/exclude-keys [keyword] yes []
:flow/short-circuit? boolean yes false