-
Notifications
You must be signed in to change notification settings - Fork 21
[:memo:] Extend python docs with lineage tracing #134
base: master
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The docs look good 👍
I spotted a few mistakes though which should be easy to fix.
Lineage Trace API | ||
================= | ||
|
||
Lineage tracing captures every instruction during runtime and store those in a DAG. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
...and stores them...
================= | ||
|
||
Lineage tracing captures every instruction during runtime and store those in a DAG. | ||
An user can retreive the string representation of a lineage trace by enabling |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A user
A *lineage item* is composed of an *id*, which is uniquely identifiable | ||
through an execution, an *opcode* for identifying the physical operation, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The id is uniquely identifiable through an execution? Do you mean, that the id uniquely identifies the lineage item throughout a single execution?
captured instruction, and a *data* attribute, which represents all necessary | ||
information. It also stores references to other *lineage items* |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The data attribute represents all necessary information to do what? What do I need the other things like name and id for, when the data attribute represents it all?
|
||
.. note:: | ||
|
||
Lineage trace is captured during execution of an instruction in SystemDS. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A lineage trace...
|
||
Lineage trace is captured during execution of an instruction in SystemDS. | ||
That means, tracing lineage for an intermediate requires SystemDS to lazily | ||
execute the DAG of operations till the desired node. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
...until the desired node has been processed.
That means, tracing lineage for an intermediate requires SystemDS to lazily | ||
execute the DAG of operations till the desired node. | ||
|
||
The example below traces lineage of an DAG of operations. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
a DAG
No description provided.