Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Request for PR: run nodes in parallel #2

Open
seshubonam opened this issue Sep 24, 2024 · 2 comments
Open

Request for PR: run nodes in parallel #2

seshubonam opened this issue Sep 24, 2024 · 2 comments

Comments

@seshubonam
Copy link
Contributor

right now, execution of nodes happens in sequence.

example: in reactflow, need to find a way to run two nodes using the previous node has finished generation of output.

here is some guidance on custom nodes in reactflow:

https://reactflow.dev/learn/customization/custom-nodes

@Aarize29
Copy link

right now, execution of nodes happens in sequence.

example: in reactflow, need to find a way to run two nodes using the previous node has finished generation of output.

here is some guidance on custom nodes in reactflow:

https://reactflow.dev/learn/customization/custom-nodes

I have worked on this type of issue in one of my project, where earlier all nodes are running sequentially, I wanted to make them run in parallel, So basically I wrote an algorithm ( just a dfs 😂 ) , in this algo what I did was, first i created one map which indicates which node is connected to which node, then I created a list in which all sequences (all paths ) from source to desination are mentioned ( I did this using Graph concept , first created an adjacency list and used dfs to map all the paths), for example [ { 0->1->2->4}, {0->1->3->4}] , here 2 and 3 are conneceted with 1 , here my source is 0 and destination is 4.

After getting the list [ { 0->1->2->4}, {0->1->3->4}], I used promise.all to execute both the path in parallel, it made my program very fast and reduced the time of execution from 17 seconds to 4 seconds

fyi :- . 4 seconds seems slow but it takes this much time because it is calling a lot of functions in the backend( quite complex project 😉).

@seshubonam
Copy link
Contributor Author

just commenting this if its useful for parallel run of nodes:

https://x.com/ryanchenkie/status/1130842992980430853?s=19

and this:

https://x.com/farzad_yz/status/1079486371230875653?t=vsegiB2C5syarIFj_NMXEg&s=19

the actual fix planned internally would be to rewrite the architecture to separate frontend and backend instead of doing it all in next js

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants