Rework Task API

[sroussey]

Rework creating a task API by putting input first, config (optional) second

In the DelayTask, the delay is not going to typically be an input from another task... it is more like a config. The ngraph node UI should treat it that way, but present it like an input.

So... is that a "config input"? Need to think about this for a beat.

Task Input/Output and DataFlow to use "all" input/output, not need to pick a specific one. This would not include the "config input"

[sroussey]

Could be a dataflow for "*" as input source.

[sroussey]

const workflow = new Workflow()
  .add(new FetchData())
  .map([new TransformData(), new BackupData()]) // Runs both in parallel
  .reduce(new SaveToDatabase()); // Merges both outputs

await workflow.run();

If I did this, could get away from always using ArrayTask, and only using it for reduce().

reduce strategies:

• array: assumes inputs and outputs are the same, and will turn them into arrays
• object_by_id: create an object where the id is the key
• object_by_type: create an object where the key is the task type (create arrays if duplicate task types or error?)
• entries_by_id: return an array [id, results],[id2, results]
• entries_by_type

[sroussey]

rename run to execute and then runFull to run.

add utility fn so a task can have its own outputrepo and if set, the runner should skip

rework how outputrepo is given to runner and passed along

[sroussey]

outputrepo should have option for compression

[sroussey]

DataFlow references to values should be "ports". It goes better with the the UI for D&D nodes. Also, we can have ports "*" for everything, and [error] for error port (though right now processing stops on an error, but maybe not in the future

[sroussey]

DataFlow should have a status so we know if it is streaming and not completed yet