Skip to content

Commit 0824934

Browse files
sryzadongjoon-hyun
authored andcommitted
[SPARK-54643][SDP][PYTHON][TESTS] Run missing pyspark pipelines tests in CI
### What changes were proposed in this pull request? Adds two pyspark pipelines tests to CI, and fixes one of them. ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Ran testes locally. ### Was this patch authored or co-authored using generative AI tooling? Closes #53393 from sryza/pipelines-test-module. Authored-by: Sandy Ryza <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
1 parent a189a45 commit 0824934

File tree

2 files changed

+4
-2
lines changed

2 files changed

+4
-2
lines changed

dev/sparktestsupport/modules.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1566,11 +1566,13 @@ def __hash__(self):
15661566
dependencies=[pyspark_core, pyspark_sql, pyspark_connect],
15671567
source_file_regexes=["python/pyspark/pipelines"],
15681568
python_test_goals=[
1569+
"pyspark.pipelines.tests.test_add_pipeline_analysis_context",
15691570
"pyspark.pipelines.tests.test_block_session_mutations",
15701571
"pyspark.pipelines.tests.test_cli",
15711572
"pyspark.pipelines.tests.test_decorators",
15721573
"pyspark.pipelines.tests.test_graph_element_registry",
15731574
"pyspark.pipelines.tests.test_init_cli",
1575+
"pyspark.pipelines.tests.test_spark_connect",
15741576
],
15751577
)
15761578

python/pyspark/pipelines/tests/test_spark_connect.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,7 @@ def mv():
5858
refresh=None,
5959
full_refresh_all=False,
6060
dry=True,
61-
storage="storage_path",
61+
storage="file:///tmp/storage_path",
6262
)
6363
handle_pipeline_events(result_iter)
6464

@@ -80,7 +80,7 @@ def st():
8080
refresh=None,
8181
full_refresh_all=False,
8282
dry=True,
83-
storage="storage_path",
83+
storage="file:///tmp/storage_path",
8484
)
8585
with self.assertRaises(AnalysisException) as context:
8686
handle_pipeline_events(result_iter)

0 commit comments

Comments
 (0)