feat: Add local python SDK replacement option via pip #150

rshade · 2025-10-16T18:35:28Z

Implements local Python SDK replacement functionality for testing Pulumi providers and programs with unpublished Python SDK builds. This feature enables developers to test against local Python SDKs using pip install -e (editable mode), following the same pattern as existing YarnLink (Node.js) and GoModReplacement (Go) features.

Key Features

Editable Mode: Uses pip install -e for symlinked/editable installation (same behavior as yarn link)
Multiple Packages: Supports installing multiple local Python packages in a single test
Absolute Paths: Converts relative paths to absolute paths for reliability
Error Handling: Clear error messages if pip install fails or paths don't exist
Environment Aware: Uses python -m pip for better virtual environment compatibility

Usage Example

import (
	"filepath"
	"testing"
	"github.com/pulumi/providertest/pulumitest"
	"github.com/pulumi/providertest/pulumitest/opttest"
)

func TestWithLocalPythonSDK(t *testing.T) {
	// Use local Python SDK build
	test := pulumitest.NewPulumiTest(t, "test_dir",
		opttest.PythonLink("../sdk/python"))

	// Or multiple packages
	test2 := pulumitest.NewPulumiTest(t, "test_dir",
		opttest.PythonLink("../sdk/python", "../other-sdk/python"))

	test.Up(t)
}

Fixes #39

Copilot

Pull Request Overview

This PR implements local Python SDK replacement functionality to enable testing Pulumi providers with unpublished Python SDK builds using pip install -e (editable mode). It follows the same architectural pattern as existing YarnLink and GoModReplacement features.

Added PythonLinks []string field to Options struct and PythonLink() function for specifying local Python package paths
Implemented pip install logic in NewStack() to execute python -m pip install -e <path> for each specified package
Added comprehensive test coverage with 6 unit tests and updated documentation with usage examples

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File	Description
pulumitest/opttest/opttest.go	Added PythonLinks field to Options struct and PythonLink() function
pulumitest/newStack.go	Implemented pip install execution logic with error handling
pulumitest/opttest/opttest_test.go	Added comprehensive unit tests for PythonLink functionality
pulumitest/README.md	Added documentation section for Python local package installation

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

pulumitest/newStack.go

…umi providers and programs with unpublished Python SDK builds. This feature enables developers to test against local Python SDKs using `pip install -e` (editable mode), following the same pattern as existing YarnLink (Node.js) and GoModReplacement (Go) features. 1. **pulumitest/opttest/opttest.go**: - Added `PythonLinks []string` field to `Options` struct to store local Python package paths - Created `PythonLink()` function to accept one or more local Python package paths - Updated `Defaults()` function to initialize `PythonLinks` slice 2. **pulumitest/newStack.go**: - Implemented pip install logic to execute `python -m pip install -e <path>` for each Python package - Uses absolute paths (consistent with GoModReplacement pattern) - Executes after YarnLink and before GoModReplacement for logical ordering - Includes proper error handling and logging 3. **pulumitest/opttest/opttest_test.go** (new file): - `TestPythonLinkOption`: Verifies single package path is appended - `TestPythonLinkMultiplePackages`: Verifies multiple package paths can be specified - `TestPythonLinkAccumulates`: Verifies packages accumulate across multiple calls - `TestDefaultsResetsPythonLinks`: Verifies Defaults() resets PythonLinks 4. **pulumitest/README.md**: - Added "Python - Local Package Installation" section - Documented PythonLink usage with examples - Followed same documentation pattern as YarnLink and GoModReplacement sections - **Editable Mode**: Uses `pip install -e` for symlinked/editable installation (same behavior as yarn link) - **Multiple Packages**: Supports installing multiple local Python packages in a single test - **Absolute Paths**: Converts relative paths to absolute paths for reliability - **Error Handling**: Clear error messages if pip install fails or paths don't exist - **Environment Aware**: Uses `python -m pip` for better virtual environment compatibility - ✅ All unit tests for PythonLink option pass (4/4 tests) - ✅ Code formatting passes (go fmt) - ✅ Code vetting passes (go vet) - ✅ Linting passes - ✅ Implementation follows existing architectural patterns for YarnLink and GoModReplacement ```go import ( "filepath" "testing" "github.com/pulumi/providertest/pulumitest" "github.com/pulumi/providertest/pulumitest/opttest" ) func TestWithLocalPythonSDK(t *testing.T) { // Use local Python SDK build test := pulumitest.NewPulumiTest(t, "test_dir", opttest.PythonLink("../sdk/python")) // Or multiple packages test2 := pulumitest.NewPulumiTest(t, "test_dir", opttest.PythonLink("../sdk/python", "../other-sdk/python")) test.Up(t) } ```

Copilot

Pull Request Overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 3 comments.

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

pulumitest/testdata/python_with_local_pkg/__main__.py

pulumitest/newStack.go

CONTRIBUTING.md

Co-authored-by: Copilot <[email protected]>

blampe

So for what it's worth I'm testing this out and it only seems to work if I also include opttest.SkipInstall(). @rshade has that been your experience as well?

With just opttest.PythonLink("../sdk/python"):

    python_test.go:14: installing packages and plugins
    python_test.go:14: failed to install packages and plugins: exit status 255
        Installing dependencies...

        Updating pip, setuptools, and wheel in virtual environment...
        Requirement already satisfied: pip in /Users/bryce/.local/share/mise/installs/python/3.11.8/lib/python3.11/site-packages (25.3)
        Requirement already satisfied: setuptools in /Users/bryce/.local/share/mise/installs/python/3.11.8/lib/python3.11/site-packages (80.9.0)
        Requirement already satisfied: wheel in /Users/bryce/.local/share/mise/installs/python/3.11.8/lib/python3.11/site-packages (0.45.1)
        Finished updating
        Installing dependencies in virtual environment...
        ERROR: Could not find a version that satisfies the requirement pulumi_provide_boilerplate>=1.0.0 (from versions: none)
        ERROR: No matching distribution found for pulumi_provide_boilerplate>=1.0.0
        error: installing dependencies: installing dependencies via '/Users/bryce/.local/share/mise/installs/python/3.11.8/bin/python3 -m pip install -r requirements.txt': exit status 1
        installing dependencies via '/Users/bryce/.local/share/mise/installs/python/3.11.8/bin/python3 -m pip install -r requirements.txt': exit status 1
        installing dependencies via '/Users/bryce/.local/share/mise/installs/python/3.11.8/bin/python3 -m pip install -r requirements.txt': exit status 1

    python_test.go:14: Skipping removal of programDir temp directories on failures: "/var/folders/kq/zdlx0fn57yl66gxktcyzylv40000gn/T/TestPython1125338078"
    python_test.go:14: To remove these directories on failures, set PULUMITEST_RETAIN_FILES_ON_FAILURE=false

Everything works as expected when I add opttest.SkipInstall().

pulumitest/newStack.go

We weren't validating that example tests still run for Node/Dotnet/Python/Java. Python needs pulumi/providertest#150. Java still needs an example program but is less of a priority.

Co-authored-by: Bryce Lampe <[email protected]>

rshade · 2025-11-18T14:32:54Z

So for what it's worth I'm testing this out and it only seems to work if I also include opttest.SkipInstall(). @rshade has that been your experience as well?

With just opttest.PythonLink("../sdk/python"):

    python_test.go:14: installing packages and plugins
    python_test.go:14: failed to install packages and plugins: exit status 255
        Installing dependencies...

        Updating pip, setuptools, and wheel in virtual environment...
        Requirement already satisfied: pip in /Users/bryce/.local/share/mise/installs/python/3.11.8/lib/python3.11/site-packages (25.3)
        Requirement already satisfied: setuptools in /Users/bryce/.local/share/mise/installs/python/3.11.8/lib/python3.11/site-packages (80.9.0)
        Requirement already satisfied: wheel in /Users/bryce/.local/share/mise/installs/python/3.11.8/lib/python3.11/site-packages (0.45.1)
        Finished updating
        Installing dependencies in virtual environment...
        ERROR: Could not find a version that satisfies the requirement pulumi_provide_boilerplate>=1.0.0 (from versions: none)
        ERROR: No matching distribution found for pulumi_provide_boilerplate>=1.0.0
        error: installing dependencies: installing dependencies via '/Users/bryce/.local/share/mise/installs/python/3.11.8/bin/python3 -m pip install -r requirements.txt': exit status 1
        installing dependencies via '/Users/bryce/.local/share/mise/installs/python/3.11.8/bin/python3 -m pip install -r requirements.txt': exit status 1
        installing dependencies via '/Users/bryce/.local/share/mise/installs/python/3.11.8/bin/python3 -m pip install -r requirements.txt': exit status 1

    python_test.go:14: Skipping removal of programDir temp directories on failures: "/var/folders/kq/zdlx0fn57yl66gxktcyzylv40000gn/T/TestPython1125338078"
    python_test.go:14: To remove these directories on failures, set PULUMITEST_RETAIN_FILES_ON_FAILURE=false

Everything works as expected when I add opttest.SkipInstall().

Let me look into that. I thought I had it working. I may have broken it after.

rshade force-pushed the python-replacement branch from 83e26e4 to 46198c3 Compare October 17, 2025 15:43

rshade requested a review from Copilot October 17, 2025 15:48

Copilot AI reviewed Oct 17, 2025

View reviewed changes

pulumitest/newStack.go Outdated Show resolved Hide resolved

rshade force-pushed the python-replacement branch from 46198c3 to 2cc0f20 Compare October 17, 2025 17:34

rshade changed the title ~~# feat: Add local python SDK replacement option via pip~~ feat: Add local python SDK replacement option via pip Oct 23, 2025

rshade force-pushed the python-replacement branch from 2cc0f20 to 65d70bc Compare October 23, 2025 20:01

rshade marked this pull request as ready for review October 23, 2025 20:02

rshade force-pushed the python-replacement branch from 65d70bc to 0b334a5 Compare October 23, 2025 20:39

rshade force-pushed the python-replacement branch from 0b334a5 to e8b12a1 Compare October 23, 2025 21:38

rshade requested review from a team, Copilot and danielrbradley October 23, 2025 21:44

Copilot AI reviewed Oct 23, 2025

View reviewed changes

pulumitest/testdata/python_with_local_pkg/__main__.py Show resolved Hide resolved

pulumitest/newStack.go Show resolved Hide resolved

CONTRIBUTING.md Outdated Show resolved Hide resolved

Apply suggestions from code review

0fa5050

Co-authored-by: Copilot <[email protected]>

blampe mentioned this pull request Nov 17, 2025

Enable more example tests pulumi/pulumi-provider-boilerplate#750

Merged

blampe reviewed Nov 17, 2025

View reviewed changes

pulumitest/newStack.go Outdated Show resolved Hide resolved

Update pulumitest/newStack.go

16bd360

Co-authored-by: Bryce Lampe <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: Add local python SDK replacement option via pip #150

feat: Add local python SDK replacement option via pip #150

Uh oh!

rshade commented Oct 16, 2025 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

blampe left a comment

Uh oh!

Uh oh!

rshade commented Nov 18, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

feat: Add local python SDK replacement option via pip #150

Are you sure you want to change the base?

feat: Add local python SDK replacement option via pip #150

Uh oh!

Conversation

rshade commented Oct 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Key Features

Usage Example

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

blampe left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

rshade commented Nov 18, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

rshade commented Oct 16, 2025 •

edited

Loading