From d3a9b2776c656077d9e20d96e67c8c1cfa24c7fe Mon Sep 17 00:00:00 2001 From: Hhhilulu <115983968+Hhhilulu@users.noreply.github.com> Date: Mon, 29 Jan 2024 16:16:15 +0800 Subject: [PATCH] [PromptFlow][Document] Modify default worker count to 4. (#1784) # Description The PR [[PromptFlow][Executor][Internal] Refine process pool.](https://github.com/microsoft/promptflow/pull/1582) modify the default worker count to 4, this PR modify related documents. # All Promptflow Contribution checklist: - [ ] **The pull request does not introduce [breaking changes].** - [ ] **CHANGELOG is updated for new features, bug fixes or other significant changes.** - [ ] **I have read the [contribution guidelines](../CONTRIBUTING.md).** - [ ] **Create an issue and link to the pull request to get dedicated review from promptflow team. Learn more: [suggested workflow](../CONTRIBUTING.md#suggested-workflow).** ## General Guidelines and Best Practices - [ ] Title of the pull request is clear and informative. - [ ] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, [see this page](https://github.com/Azure/azure-powershell/blob/master/documentation/development-docs/cleaning-up-commits.md). ### Testing Guidelines - [ ] Pull request includes test coverage for the included changes. --- docs/how-to-guides/faq.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/how-to-guides/faq.md b/docs/how-to-guides/faq.md index da6a23fe0a3..a0096d56062 100644 --- a/docs/how-to-guides/faq.md +++ b/docs/how-to-guides/faq.md @@ -82,15 +82,15 @@ Currently, promptflow supports the following environment variables: **PF_WORKER_COUNT** -Valid for batch run only. The number of workers to use for parallel execution of the Flow. +Effective for batch run only, count of parallel workers in batch run execution. -Default value is 16. If you have large number of batch run date row count, and want more efficiency, you can increase the PF_WORKER_COUNT to improve the batch run concurrency, make it run faster. +The default value is 4 (was 16 when promptflow<1.4.0) -When you modify the concurrency, please consider 2 points: +Please take the following points into consideration when changing it: -First, the concurrency should be not bigger than your batch run data row count. If not, meaning if the concurrency is bigger, it will run slower due to the time taken for process startup and shutdown. +1. The concurrency should not exceed the total data rows count. Otherwise, the execution may slow down due to additional time spent on process startup and shutdown. -Second, your batch run risks to fail due to rate limit of your LLM endpoint, in this case you need to set up PF_WORKER_COUNT to a smaller number. Take Azure OpenAI endpoint as example, you can go to Azure OpenAI Studio, navigate to Deployment tab, check out the capacity of your endpoints. Then you can refer to this expression to set up the concurrency. +2. High parallelism may cause the underlying API call to reach the rate limit of your LLM endpoint. In which case you can decrease the `PF_WORKER_COUNT` or increase the rate limit. Please refer to [this doc](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/quota) on quota management. Then you can refer to this expression to set up the concurrency. ``` PF_WORKER_COUNT <= TPM * duration_seconds / token_count / 60