promptflow-release-testing-matrix #58
zhengfeiwang
This run and associated checks have been archived and are scheduled for deletion.
Learn more about checks retention
promptflow-release-testing-matrix.yml
on: workflow_dispatch
Matrix: all_tests
id
2s
Publish Tests Results
26s
Annotations
12 errors and 40 warnings
|
all_tests (windows-latest, executor, 3.8)
Process completed with exit code 1.
|
|
all_tests (windows-latest, executor, 3.10)
Process completed with exit code 1.
|
|
all_tests (windows-latest, executor, 3.11)
Process completed with exit code 1.
|
|
all_tests (windows-latest, sdk-cli, 3.10)
Process completed with exit code 1.
|
|
all_tests (windows-latest, sdk-cli, 3.9)
The job running on runner GitHub Actions 98 has exceeded the maximum execution time of 360 minutes.
|
|
all_tests (windows-latest, sdk-cli, 3.9)
The operation was canceled.
|
|
all_tests (windows-latest, sdk-cli, 3.8)
The job running on runner GitHub Actions 102 has exceeded the maximum execution time of 360 minutes.
|
|
all_tests (windows-latest, sdk-cli, 3.8)
The operation was canceled.
|
|
all_tests (windows-latest, sdk-cli, 3.11)
The job running on runner GitHub Actions 100 has exceeded the maximum execution time of 360 minutes.
|
|
all_tests (windows-latest, sdk-cli, 3.11)
The operation was canceled.
|
|
All 3 runs with error: tests.executor.unittests.contracts.test_tool:
/#L0
artifacts/Test Results (Python 3.8) (OS macos-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.8) (OS ubuntu-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.8) (OS windows-latest)/test-results-executor.xml [took 0s]
|
|
All 3 runs with error: tests.executor.unittests.executor.test_tool_resolver:
/#L0
artifacts/Test Results (Python 3.8) (OS macos-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.8) (OS ubuntu-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.8) (OS windows-latest)/test-results-executor.xml [took 0s]
|
|
all_tests (windows-latest, sdk-cli, 3.9)
No files were found with the provided path: D:\a\promptflow\promptflow/*.xml. No artifacts will be uploaded.
|
|
all_tests (windows-latest, sdk-cli, 3.8)
No files were found with the provided path: D:\a\promptflow\promptflow/*.xml. No artifacts will be uploaded.
|
|
all_tests (windows-latest, sdk-cli, 3.11)
No files were found with the provided path: D:\a\promptflow\promptflow/*.xml. No artifacts will be uploaded.
|
|
1 out of 12 runs failed: test_bulk_run_input_type_invalid[simple_flow_with_python_tool-batch_input1-The input for flow is incor…or adjust the input value in your input data.-InputTypeError] (tests.executor.e2etests.test_executor_validation.TestValidation):
tests.executor.e2etests.test_executor_validation.TestValidation#L0
artifacts/Test Results (Python 3.11) (OS windows-latest)/test-results-executor.xml [took 4s]
|
|
3 out of 12 runs failed: test_single_node_input_type_invalid[path_root1-simple_flow_with_python_tool-divide_num-line_input1-Inp…is not type 'int'. Please review and rectify the input data.] (tests.executor.e2etests.test_executor_validation.TestValidation):
tests.executor.e2etests.test_executor_validation.TestValidation#L0
artifacts/Test Results (Python 3.11) (OS macos-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS ubuntu-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS windows-latest)/test-results-executor.xml [took 0s]
|
|
3 out of 12 runs failed: test_gen_tool_by_source_error[tool_source2-custom_llm-NotSupported-The tool type custom_llm is current…. If you need further assistance, kindly contact support.] (tests.executor.unittests._core.test_tools_manager.TestToolsManager):
tests.executor.unittests._core.test_tools_manager.TestToolsManager#L0
artifacts/Test Results (Python 3.11) (OS macos-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS ubuntu-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS windows-latest)/test-results-executor.xml [took 0s]
|
|
3 out of 12 runs failed: test_get_user_execution_error_info[raise_tool_execution_error-ToolExecutionError] (tests.executor.unittests._utils.test_exception_utils.TestErrorResponse):
tests.executor.unittests._utils.test_exception_utils.TestErrorResponse#L0
artifacts/Test Results (Python 3.11) (OS macos-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS ubuntu-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS windows-latest)/test-results-executor.xml [took 0s]
|
|
3 out of 12 runs failed: test_tool_execution_error (tests.executor.unittests._utils.test_exception_utils.TestExceptions):
tests.executor.unittests._utils.test_exception_utils.TestExceptions#L0
artifacts/Test Results (Python 3.11) (OS macos-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS ubuntu-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS windows-latest)/test-results-executor.xml [took 0s]
|
|
3 out of 12 runs failed: test_additional_info (tests.executor.unittests._utils.test_exception_utils.TestExceptions):
tests.executor.unittests._utils.test_exception_utils.TestExceptions#L0
artifacts/Test Results (Python 3.11) (OS macos-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS ubuntu-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS windows-latest)/test-results-executor.xml [took 0s]
|
|
3 out of 12 runs failed: test_generate_tool_meta_dict_by_file_exception[MetaFileNotFound] (tests.executor.unittests._utils.test_generate_tool_meta_utils.TestToolMetaUtils):
tests.executor.unittests._utils.test_generate_tool_meta_utils.TestToolMetaUtils#L0
artifacts/Test Results (Python 3.11) (OS macos-latest)/test-results-executor.xml [took 1s]
artifacts/Test Results (Python 3.11) (OS ubuntu-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS windows-latest)/test-results-executor.xml [took 3s]
|
|
3 out of 12 runs failed: test_generate_tool_meta_dict_by_file_exception[MetaFileReadError] (tests.executor.unittests._utils.test_generate_tool_meta_utils.TestToolMetaUtils):
tests.executor.unittests._utils.test_generate_tool_meta_utils.TestToolMetaUtils#L0
artifacts/Test Results (Python 3.11) (OS macos-latest)/test-results-executor.xml [took 1s]
artifacts/Test Results (Python 3.11) (OS ubuntu-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS windows-latest)/test-results-executor.xml [took 3s]
|
|
3 out of 12 runs failed: test_generate_tool_meta_dict_by_file_exception[NotSupported] (tests.executor.unittests._utils.test_generate_tool_meta_utils.TestToolMetaUtils):
tests.executor.unittests._utils.test_generate_tool_meta_utils.TestToolMetaUtils#L0
artifacts/Test Results (Python 3.11) (OS macos-latest)/test-results-executor.xml [took 1s]
artifacts/Test Results (Python 3.11) (OS ubuntu-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS windows-latest)/test-results-executor.xml [took 3s]
|
|
3 out of 12 runs failed: test_custom_llm_meta[JinjaParsingError_Code] (tests.executor.unittests._utils.test_generate_tool_meta_utils.TestToolMetaUtils):
tests.executor.unittests._utils.test_generate_tool_meta_utils.TestToolMetaUtils#L0
artifacts/Test Results (Python 3.11) (OS macos-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS ubuntu-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS windows-latest)/test-results-executor.xml [took 0s]
|
|
3 out of 12 runs failed: test_custom_llm_meta[JinjaParsingError_File] (tests.executor.unittests._utils.test_generate_tool_meta_utils.TestToolMetaUtils):
tests.executor.unittests._utils.test_generate_tool_meta_utils.TestToolMetaUtils#L0
artifacts/Test Results (Python 3.11) (OS macos-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS ubuntu-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS windows-latest)/test-results-executor.xml [took 0s]
|
|
3 out of 12 runs failed: test_custom_prompt_meta[JinjaParsingError_Code] (tests.executor.unittests._utils.test_generate_tool_meta_utils.TestToolMetaUtils):
tests.executor.unittests._utils.test_generate_tool_meta_utils.TestToolMetaUtils#L0
artifacts/Test Results (Python 3.11) (OS macos-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS ubuntu-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS windows-latest)/test-results-executor.xml [took 0s]
|
|
3 out of 12 runs failed: test_custom_prompt_meta[JinjaParsingError_File] (tests.executor.unittests._utils.test_generate_tool_meta_utils.TestToolMetaUtils):
tests.executor.unittests._utils.test_generate_tool_meta_utils.TestToolMetaUtils#L0
artifacts/Test Results (Python 3.11) (OS macos-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS ubuntu-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS windows-latest)/test-results-executor.xml [took 0s]
|
|
3 out of 12 runs failed: test_resolve_tool_error (tests.executor.unittests.executor.test_errors):
tests.executor.unittests.executor.test_errors#L0
artifacts/Test Results (Python 3.11) (OS macos-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS ubuntu-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS windows-latest)/test-results-executor.xml [took 0s]
|
|
3 out of 12 runs failed: test_resolve_flow_inputs_type_json_error_for_list_type[flow_with_list_input-inputs0-None-InputParseErr…Please make sure your inputs are properly formatted.] (tests.executor.unittests.executor.test_flow_validator.TestFlowValidator):
tests.executor.unittests.executor.test_flow_validator.TestFlowValidator#L0
artifacts/Test Results (Python 3.11) (OS macos-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS ubuntu-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS windows-latest)/test-results-executor.xml [took 0s]
|
|
3 out of 12 runs failed: test_resolve_flow_inputs_type_json_error_for_list_type[flow_with_list_input-inputs1-0-InputParseError-…Please make sure your inputs are properly formatted.] (tests.executor.unittests.executor.test_flow_validator.TestFlowValidator):
tests.executor.unittests.executor.test_flow_validator.TestFlowValidator#L0
artifacts/Test Results (Python 3.11) (OS macos-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS ubuntu-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS windows-latest)/test-results-executor.xml [took 0s]
|
|
3 out of 12 runs failed: test_resolve_aggregated_flow_inputs_type_error[inputs0-The input for flow is incorrect. The value for …t type or adjust the input value in your input data.] (tests.executor.unittests.executor.test_flow_validator.TestFlowValidator):
tests.executor.unittests.executor.test_flow_validator.TestFlowValidator#L0
artifacts/Test Results (Python 3.11) (OS macos-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS ubuntu-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS windows-latest)/test-results-executor.xml [took 0s]
|
|
3 out of 12 runs failed: test_parse_input_value_error[str-int-None-InputTypeError-The input for flow is incorrect. The value fo…t type or adjust the input value in your input data.] (tests.executor.unittests.executor.test_flow_validator.TestFlowValidator):
tests.executor.unittests.executor.test_flow_validator.TestFlowValidator#L0
artifacts/Test Results (Python 3.11) (OS macos-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS ubuntu-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS windows-latest)/test-results-executor.xml [took 0s]
|
|
3 out of 12 runs failed: test_parse_input_value_error[['1', '2']-list-None-InputParseError-Failed to parse the flow input. The …Please make sure your inputs are properly formatted.] (tests.executor.unittests.executor.test_flow_validator.TestFlowValidator):
tests.executor.unittests.executor.test_flow_validator.TestFlowValidator#L0
artifacts/Test Results (Python 3.11) (OS macos-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS ubuntu-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS windows-latest)/test-results-executor.xml [took 0s]
|
|
3 out of 12 runs failed: test_parse_input_value_error[str-int-10-InputTypeError-The input for flow is incorrect. The value for …t type or adjust the input value in your input data.] (tests.executor.unittests.executor.test_flow_validator.TestFlowValidator):
tests.executor.unittests.executor.test_flow_validator.TestFlowValidator#L0
artifacts/Test Results (Python 3.11) (OS macos-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS ubuntu-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS windows-latest)/test-results-executor.xml [took 0s]
|
|
3 out of 12 runs failed: test_parse_input_value_error[['1', '2']-list-10-InputParseError-Failed to parse the flow input. The va…Please make sure your inputs are properly formatted.] (tests.executor.unittests.executor.test_flow_validator.TestFlowValidator):
tests.executor.unittests.executor.test_flow_validator.TestFlowValidator#L0
artifacts/Test Results (Python 3.11) (OS macos-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS ubuntu-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS windows-latest)/test-results-executor.xml [took 0s]
|
|
3 out of 9 runs failed: test_ensure_node_inputs_type (tests.executor.unittests.executor.test_tool_resolver.TestToolResolver):
tests.executor.unittests.executor.test_tool_resolver.TestToolResolver#L0
artifacts/Test Results (Python 3.11) (OS macos-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS ubuntu-latest)/test-results-executor.xml [took 0s]
artifacts/Test Results (Python 3.11) (OS windows-latest)/test-results-executor.xml [took 0s]
|
|
1 out of 12 runs failed: test_healthy_ensured_process (tests.executor.unittests.processpool.test_healthy_ensured_process.TestHealthyEnsuredProcess):
tests.executor.unittests.processpool.test_healthy_ensured_process.TestHealthyEnsuredProcess#L0
artifacts/Test Results (Python 3.10) (OS windows-latest)/test-results-executor.xml [took 6s]
|
|
1 out of 12 runs failed: test_unhealthy_process (tests.executor.unittests.processpool.test_healthy_ensured_process.TestHealthyEnsuredProcess):
tests.executor.unittests.processpool.test_healthy_ensured_process.TestHealthyEnsuredProcess#L0
artifacts/Test Results (Python 3.11) (OS windows-latest)/test-results-executor.xml [took 30s]
|
|
1 out of 9 runs failed: test_basic_flow_run_bulk_without_env (tests.sdk_cli_azure_test.e2etests.test_cli_with_azure.TestCliWithAzure):
tests.sdk_cli_azure_test.e2etests.test_cli_with_azure.TestCliWithAzure#L0
artifacts/Test Results (Python 3.10) (OS macos-latest)/test-results-sdk-cli.xml [took 1m 33s]
|
|
2 out of 9 runs failed: test_run_with_remote_data (tests.sdk_cli_azure_test.e2etests.test_cli_with_azure.TestCliWithAzure):
tests.sdk_cli_azure_test.e2etests.test_cli_with_azure.TestCliWithAzure#L0
artifacts/Test Results (Python 3.10) (OS windows-latest)/test-results-sdk-cli.xml [took 1m 32s]
artifacts/Test Results (Python 3.9) (OS macos-latest)/test-results-sdk-cli.xml [took 2m 56s]
|
|
5 out of 9 runs failed: test_basic_evaluation (tests.sdk_cli_azure_test.e2etests.test_run_operations.TestFlowRun):
tests.sdk_cli_azure_test.e2etests.test_run_operations.TestFlowRun#L0
artifacts/Test Results (Python 3.11) (OS macos-latest)/test-results-sdk-cli.xml [took 1m 25s]
artifacts/Test Results (Python 3.11) (OS ubuntu-latest)/test-results-sdk-cli.xml [took 10m 0s]
artifacts/Test Results (Python 3.8) (OS macos-latest)/test-results-sdk-cli.xml [took 5m 46s]
artifacts/Test Results (Python 3.8) (OS ubuntu-latest)/test-results-sdk-cli.xml [took 10m 0s]
artifacts/Test Results (Python 3.9) (OS macos-latest)/test-results-sdk-cli.xml [took 1m 38s]
|
|
1 out of 9 runs failed: test_run_with_env_overwrite (tests.sdk_cli_azure_test.e2etests.test_run_operations.TestFlowRun):
tests.sdk_cli_azure_test.e2etests.test_run_operations.TestFlowRun#L0
artifacts/Test Results (Python 3.10) (OS ubuntu-latest)/test-results-sdk-cli.xml [took 1m 31s]
|
|
1 out of 9 runs failed: test_run_display_name_with_macro (tests.sdk_cli_azure_test.e2etests.test_run_operations.TestFlowRun):
tests.sdk_cli_azure_test.e2etests.test_run_operations.TestFlowRun#L0
artifacts/Test Results (Python 3.10) (OS ubuntu-latest)/test-results-sdk-cli.xml [took 1m 36s]
|
|
1 out of 9 runs failed: test_run_with_connection_overwrite (tests.sdk_cli_azure_test.e2etests.test_run_operations.TestFlowRun):
tests.sdk_cli_azure_test.e2etests.test_run_operations.TestFlowRun#L0
artifacts/Test Results (Python 3.9) (OS ubuntu-latest)/test-results-sdk-cli.xml [took 1m 26s]
|
|
2 out of 9 runs failed: test_run_bulk_from_yaml (tests.sdk_cli_azure_test.e2etests.test_run_operations.TestFlowRun):
tests.sdk_cli_azure_test.e2etests.test_run_operations.TestFlowRun#L0
artifacts/Test Results (Python 3.10) (OS macos-latest)/test-results-sdk-cli.xml [took 17s]
artifacts/Test Results (Python 3.9) (OS ubuntu-latest)/test-results-sdk-cli.xml [took 2m 30s]
|
|
2 out of 9 runs failed: test_archive_and_restore_run (tests.sdk_cli_azure_test.e2etests.test_run_operations.TestFlowRun):
tests.sdk_cli_azure_test.e2etests.test_run_operations.TestFlowRun#L0
artifacts/Test Results (Python 3.11) (OS macos-latest)/test-results-sdk-cli.xml [took 4s]
artifacts/Test Results (Python 3.8) (OS ubuntu-latest)/test-results-sdk-cli.xml [took 4s]
|
|
3 out of 9 runs failed: test_run_with_additional_includes (tests.sdk_cli_azure_test.e2etests.test_run_operations.TestFlowRun):
tests.sdk_cli_azure_test.e2etests.test_run_operations.TestFlowRun#L0
artifacts/Test Results (Python 3.11) (OS ubuntu-latest)/test-results-sdk-cli.xml [took 10m 0s]
artifacts/Test Results (Python 3.8) (OS macos-latest)/test-results-sdk-cli.xml [took 5m 46s]
artifacts/Test Results (Python 3.9) (OS ubuntu-latest)/test-results-sdk-cli.xml [took 1m 30s]
|
|
1 out of 9 runs failed: test_run_without_dump (tests.sdk_cli_azure_test.e2etests.test_run_operations.TestFlowRun):
tests.sdk_cli_azure_test.e2etests.test_run_operations.TestFlowRun#L0
artifacts/Test Results (Python 3.11) (OS macos-latest)/test-results-sdk-cli.xml [took 1m 27s]
|
|
1 out of 9 runs failed: test_run_with_remote_data (tests.sdk_cli_azure_test.e2etests.test_run_operations.TestFlowRun):
tests.sdk_cli_azure_test.e2etests.test_run_operations.TestFlowRun#L0
artifacts/Test Results (Python 3.11) (OS macos-latest)/test-results-sdk-cli.xml [took 1m 58s]
|
|
2 out of 9 runs failed: test_get_detail_against_partial_fail_run (tests.sdk_cli_azure_test.e2etests.test_run_operations.TestFlowRun):
tests.sdk_cli_azure_test.e2etests.test_run_operations.TestFlowRun#L0
artifacts/Test Results (Python 3.11) (OS macos-latest)/test-results-sdk-cli.xml [took 1m 33s]
artifacts/Test Results (Python 3.8) (OS macos-latest)/test-results-sdk-cli.xml [took 10m 0s]
|
|
2 out of 9 runs failed: test_input_mapping_with_dict (tests.sdk_cli_azure_test.e2etests.test_run_operations.TestFlowRun):
tests.sdk_cli_azure_test.e2etests.test_run_operations.TestFlowRun#L0
artifacts/Test Results (Python 3.10) (OS macos-latest)/test-results-sdk-cli.xml [took 59s]
artifacts/Test Results (Python 3.8) (OS macos-latest)/test-results-sdk-cli.xml [took 1m 29s]
|
Artifacts
Produced during runtime
| Name | Size | |
|---|---|---|
|
Test Results (Python 3.10) (OS macos-latest)
Expired
|
162 KB |
|
|
Test Results (Python 3.10) (OS ubuntu-latest)
Expired
|
162 KB |
|
|
Test Results (Python 3.10) (OS windows-latest)
Expired
|
153 KB |
|
|
Test Results (Python 3.11) (OS macos-latest)
Expired
|
267 KB |
|
|
Test Results (Python 3.11) (OS ubuntu-latest)
Expired
|
234 KB |
|
|
Test Results (Python 3.11) (OS windows-latest)
Expired
|
237 KB |
|
|
Test Results (Python 3.8) (OS macos-latest)
Expired
|
167 KB |
|
|
Test Results (Python 3.8) (OS ubuntu-latest)
Expired
|
145 KB |
|
|
Test Results (Python 3.8) (OS windows-latest)
Expired
|
94.8 KB |
|
|
Test Results (Python 3.9) (OS macos-latest)
Expired
|
163 KB |
|
|
Test Results (Python 3.9) (OS ubuntu-latest)
Expired
|
164 KB |
|
|
Test Results (Python 3.9) (OS windows-latest)
Expired
|
92.9 KB |
|