Pipeline Execution Diagram
- Scheduler object creates Execution object when it is "due"
- Scheduler "context" -- it is a template, when rendered, it becomes a JSON object
- When rendering the scheduler context, "due" is a variable represent the execution "due" time (a datetime object)
- Execution object has "context", it is rendered from Scheduler objects context and it is a JSON object
- Pipeline can access the execution's context.
- Pipeline's task's parameter is template (jinja template), execution's context will be used to render the template
- Each Pipeline can declare asset dependency, execution will only invoke pipeline when dependent assets are present in the Data Manager.