Skip to content

[Internal][Executor] Run exec_line concurrently and refine cancel logic in batch engine #3541

[Internal][Executor] Run exec_line concurrently and refine cancel logic in batch engine

[Internal][Executor] Run exec_line concurrently and refine cancel logic in batch engine #3541

GitHub Actions / SDK CLI Test Result [devs/peiwen/refine_cancel](https://github.com/microsoft/promptflow/actions/workflows/promptflow-sdk-cli-test.yml?query=branch:devs/peiwen/refine_cancel++) failed Dec 7, 2023 in 0s

1 fail, 5 skipped, 298 pass in 3m 39s

    2 files  ±0      2 suites  ±0   3m 39s ⏱️ ±0s
304 tests ±0  298 ✔️ ±0    5 💤 ±0  1 ±0 
608 runs  ±0  596 ✔️ ±0  10 💤 ±0  2 ±0 

Results for commit a68c061. ± Comparison against earlier commit 0c4d59f.

Annotations

Check warning on line 0 in tests.sdk_cli_test.e2etests.test_flow_run.TestFlowRun

See this annotation in the file changed.

@github-actions github-actions / SDK CLI Test Result [devs/peiwen/refine_cancel](https://github.com/microsoft/promptflow/actions/workflows/promptflow-sdk-cli-test.yml?query=branch:devs/peiwen/refine_cancel++)

All 2 runs failed: test_csharp_flow (tests.sdk_cli_test.e2etests.test_flow_run.TestFlowRun)

artifacts/Test Results (Python 3.8) (OS ubuntu-latest)/test-results-sdk-cli.xml [took 0s]
artifacts/Test Results (Python 3.9) (OS ubuntu-latest)/test-results-sdk-cli.xml [took 0s]
Raw output
FileNotFoundError: flow path D:/PromptflowCS/src/PromptflowCSharp/Sample/Basic/bin/Debug/net6.0 does not exist
self = <sdk_cli_test.e2etests.test_flow_run.TestFlowRun object at 0x7f83790b03d0>
pf = <promptflow._sdk._pf_client.PFClient object at 0x7f8378e5bac0>

    def test_csharp_flow(self, pf):
        # flow_path = "C:/Users/peiwengao/source/repos/HelloWorldFlow/bin/Debug/net6.0"
        flow_path = "D:/PromptflowCS/src/PromptflowCSharp/Sample/Basic/bin/Debug/net6.0"
        data_path = "D:/small_inputs.jsonl"
    
>       result = pf.run(
            flow=flow_path,
            data=data_path,
            column_mapping={
                "question": "${data.question}",
            },
        )

tests/sdk_cli_test/e2etests/test_flow_run.py:1046: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <promptflow._sdk._pf_client.PFClient object at 0x7f8378e5bac0>
flow = 'D:/PromptflowCS/src/PromptflowCSharp/Sample/Basic/bin/Debug/net6.0'
data = 'D:/small_inputs.jsonl', run = None
column_mapping = {'question': '${data.question}'}, variant = None
connections = None, environment_variables = None, name = None
display_name = None

    def run(
        self,
        flow: Union[str, PathLike],
        *,
        data: Union[str, PathLike] = None,
        run: Union[str, Run] = None,
        column_mapping: dict = None,
        variant: str = None,
        connections: dict = None,
        environment_variables: dict = None,
        name: str = None,
        display_name: str = None,
        tags: Dict[str, str] = None,
        **kwargs,
    ) -> Run:
        """Run flow against provided data or run.
    
        .. note::
            At least one of the ``data`` or ``run`` parameters must be provided.
    
        .. admonition:: Column_mapping
    
            Column mapping is a mapping from flow input name to specified values.
            If specified, the flow will be executed with provided value for specified inputs.
            The value can be:
    
            - from data:
                - ``data.col1``
            - from run:
                - ``run.inputs.col1``: if need reference run's inputs
                - ``run.output.col1``: if need reference run's outputs
            - Example:
                - ``{"ground_truth": "${data.answer}", "prediction": "${run.outputs.answer}"}``
    
        :param flow: Path to the flow directory to run evaluation.
        :type flow: Union[str, PathLike]
        :param data: Pointer to the test data (of variant bulk runs) for eval runs.
        :type data: Union[str, PathLike]
        :param run: Flow run ID or flow run. This parameter helps keep lineage between
            the current run and variant runs. Batch outputs can be
            referenced as ``${run.outputs.col_name}`` in inputs_mapping.
        :type run: Union[str, ~promptflow.entities.Run]
        :param column_mapping: Define a data flow logic to map input data.
        :type column_mapping: Dict[str, str]
        :param variant: Node & variant name in the format of ``${node_name.variant_name}``.
            The default variant will be used if not specified.
        :type variant: str
        :param connections: Overwrite node level connections with provided values.
            Example: ``{"node1": {"connection": "new_connection", "deployment_name": "gpt-35-turbo"}}``
        :type connections: Dict[str, Dict[str, str]]
        :param environment_variables: Environment variables to set by specifying a property path and value.
            Example: ``{"key1": "${my_connection.api_key}", "key2"="value2"}``
            The value reference to connection keys will be resolved to the actual value,
            and all environment variables specified will be set into os.environ.
        :type environment_variables: Dict[str, str]
        :param name: Name of the run.
        :type name: str
        :param display_name: Display name of the run.
        :type display_name: str
        :param tags: Tags of the run.
        :type tags: Dict[str, str]
        :return: Flow run info.
        :rtype: ~promptflow.entities.Run
        """
        if not os.path.exists(flow):
>           raise FileNotFoundError(f"flow path {flow} does not exist")
E           FileNotFoundError: flow path D:/PromptflowCS/src/PromptflowCSharp/Sample/Basic/bin/Debug/net6.0 does not exist

promptflow/_sdk/_pf_client.py:109: FileNotFoundError