We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Describe the bug Flow considers only the flow file to create the hash used for caching: https://github.com/microsoft/promptflow/blob/promptflow_1.16.1/src/promptflow-devkit/promptflow/_sdk/entities/_flows/base.py#L166-L169.
If we have two flows with the exactly same flow.dag.yaml, but different tools, it may run the wrong flow.
flow.dag.yaml
How To Reproduce the bug (see the test reproducing the issue in Additional context section below)
Current behavior: Only the first loaded flow runs twice. Expected behavior: Two different flows are loaded and run. Running Information:
{ "promptflow": "1.16.1", "promptflow-core": "1.16.1", "promptflow-devkit": "1.16.1", "promptflow-tracing": "1.16.1" } Executable '/home/user/workspace/test-promptflow/.venv/bin/python' Python (Linux) 3.12.1 (main, Dec 12 2023, 10:34:48) [GCC 10.2.1 20210110]
Debian GNU/Linux 11 (bullseye)
3.12.1
Additional context Test reproducing the issue:
import tempfile from pathlib import Path from textwrap import dedent import pytest from promptflow.client import load_flow @pytest.fixture def flow_yaml_content(): return dedent( """ $schema: https://azuremlschemas.azureedge.net/promptflow/latest/Flow.schema.json inputs: text: type: string outputs: text: type: string reference: ${transform} nodes: - name: transform type: python source: type: code path: transform.py inputs: text: ${inputs.text}""" ) @pytest.fixture def lower_flow_dir(flow_yaml_content): with tempfile.TemporaryDirectory() as lower_flow_dir: lower_flow_dir = Path(lower_flow_dir) with open(lower_flow_dir / 'flow.dag.yaml', 'w') as f: f.write(flow_yaml_content) with open(lower_flow_dir / "transform.py", "w") as f: f.write( dedent( """ from promptflow import tool @tool def lower(text): return text.lower()""" ) ) yield lower_flow_dir @pytest.fixture def upper_flow_dir(flow_yaml_content): with tempfile.TemporaryDirectory() as upper_flow_dir: upper_flow_dir = Path(upper_flow_dir) with open(upper_flow_dir / 'flow.dag.yaml', 'w') as f: f.write(flow_yaml_content) with open(upper_flow_dir / "transform.py", "w") as f: f.write( dedent( """ from promptflow import tool @tool def upper(text): return text.upper()""" ) ) yield upper_flow_dir def test_ignore_cache_when_tools_are_different(lower_flow_dir, upper_flow_dir): assert load_flow(lower_flow_dir).invoke({"text": "TEST"}).output == {"text": "test"} assert load_flow(upper_flow_dir).invoke({"text": "test"}).output == {"text": "TEST"}
___________________________ test_ignore_cache_when_tools_are_different ___________________________ lower_flow_dir = PosixPath('/tmp/tmpiizohoz4'), upper_flow_dir = PosixPath('/tmp/tmp1wl2g71f') def test_ignore_cache_when_tools_are_different(lower_flow_dir, upper_flow_dir): assert load_flow(lower_flow_dir).invoke({"text": "TEST"}).output == {"text": "test"} > assert load_flow(upper_flow_dir).invoke({"text": "test"}).output == {"text": "TEST"} E AssertionError: assert {'text': 'test'} == {'text': 'TEST'} E E Differing items: E {'text': 'test'} != {'text': 'TEST'} E Use -v to get more diff test_promptflow.py:79: AssertionError
The text was updated successfully, but these errors were encountered:
No branches or pull requests
Describe the bug
Flow considers only the flow file to create the hash used for caching: https://github.com/microsoft/promptflow/blob/promptflow_1.16.1/src/promptflow-devkit/promptflow/_sdk/entities/_flows/base.py#L166-L169.
If we have two flows with the exactly same
flow.dag.yaml
, but different tools, it may run the wrong flow.How To Reproduce the bug
(see the test reproducing the issue in Additional context section below)
flow.dag.yaml
but different tools.Current behavior: Only the first loaded flow runs twice.
Expected behavior: Two different flows are loaded and run.
Running Information:
Debian GNU/Linux 11 (bullseye)
3.12.1
Additional context
Test reproducing the issue:
The text was updated successfully, but these errors were encountered: