Graph Partitioning

[kckeiks]

Docs say that a graph is partitioned https://onnxruntime.ai/docs/reference/high-level-design.html#high-level-system-architecture. What is the algorithm used to partition the graph? Could you please describe how that works?

Thank you in advance.

[kckeiks]

Anyone? Where in the code could I find the answer?

[pranavsharma]

The partitioning algorithm is roughly as follows. (I'm intentionally skipping the finer details here). Code is here https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/core/framework/graph_partitioner.cc#L940.

for (e : prioritized list of EPs)
  e.GetCapability(onnx_graph) // gives this EP chance to mark nodes or return subgraphs it wants to claim (run)

The priority of EPs is the order in which uses register them with the session.
Unmarked nodes in the graph are assigned to CPU by default.

[kckeiks]

The partitioning algorithm is roughly as follows. (I'm intentionally skipping the finer details here). Code is here https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/core/framework/graph_partitioner.cc#L940.

for (e : prioritized list of EPs)
  e.GetCapability(onnx_graph) // gives this EP chance to mark nodes or return subgraphs it wants to claim (run)

The priority of EPs is the order in which uses register them with the session. Unmarked nodes in the graph are assigned to CPU by default.

This is great, thank you!