Python Test Refactoring

.github/python.instructions.md

@@ -48,14 +48,21 @@ applyTo: '**/*.py'
   2. Specific type/constant imports (e.g., `from apimtypes import INFRASTRUCTURE`)
   3. Specific function imports (e.g., `from console import print_error`)
 
-## Linting (pylint)
+## Code Quality Checklist
 
-- Respect the repository pylint configuration at `tests/python/.pylintrc`.
-- When changing Python code, run pylint and ensure changes do not worsen the pylint rating unexpectedly.
-- Prefer fixing root causes (e.g., import structure, error handling) over suppressions.
+Before completing any Python code changes, verify:
+
+- [ ] All pylint warnings and errors are resolved (`pylint --rcfile=tests/python/.pylintrc <file>`)
+- [ ] Code follows PEP 8 and the style guidelines in this file
+- [ ] Import statements for modules within this repo are placed last in the imports and are grouped with the `# APIM Samples imports` header
+- [ ] Type hints are present where appropriate
+- [ ] No unnecessary comments; docstrings are present for functions and classes
+- [ ] Edge cases and error handling are implemented
+- [ ] Prefer fixing root causes (e.g., import structure, error handling) over suppressions.
 
 ## Testing
 
+- Aim for 90+% code coverage for each file.
 - Add or update pytest unit tests when changing behavior.
 - Prefer focused tests for the code being changed.
 - Avoid tests that require live Azure access; mock Azure CLI interactions and `azure_resources` helpers.

.gitignore

@@ -26,6 +26,10 @@ labs-in-progress/
 
 # Coverage data and reports
 .coverage
+.coverage.*
+coverage.xml
+coverage.json
+htmlcov/
 tests/python/htmlcov/
 
 # Pylint reports
@@ -40,3 +44,5 @@ Test-Matrix.html
 
 $JsonReport
 $TextReport
+$JsonReportRelative
+$TextReportRelative

shared/python/infrastructures.py

@@ -145,7 +145,7 @@ def _create_keyvault(self, key_vault_name: str) -> bool:
                 print_error('Failed to assign Key Vault Certificates Officer role to current user.\nThis is an RBAC permission issue - verify your account has sufficient permissions.')
                 return False
 
-            print_ok(' Assigned Key Vault Certificates Officer role to current user')
+            print_ok('Assigned Key Vault Certificates Officer role to current user')
 
             # Brief wait for role assignment propagation
             print_plain('⏳ Waiting for role assignment propagation (15 seconds)...')

shared/python/utils.py

@@ -117,7 +117,6 @@ def __init__(self, rg_location: str, deployment: INFRASTRUCTURE, index: int, api
         print_val('Infrastructure', self.deployment.value)
         print_val('Index', self.index)
         print_val('APIM SKU', self.apim_sku.value)
-        print_plain('')
 
     # ------------------------------
     #    PUBLIC METHODS

tests/README.md

@@ -80,12 +80,21 @@ Run tests separately when you only need test execution:
 
 Both scripts:
 - Run all tests in `tests/python` using pytest
-- Generate a code coverage report (HTML output in `tests/python/htmlcov`)
-- Store the raw coverage data in `tests/python/.coverage`
+- Generate code coverage reports:
+  - HTML: `htmlcov/index.html` (at repository root)
+  - XML: `coverage.xml` (for VS Code integration)
+  - JSON: `coverage.json`
+- Store the raw coverage data in `.coverage` (at repository root)
 
 #### Viewing Coverage Reports
 
-After running tests, open `tests/python/htmlcov/index.html` in your browser to view detailed coverage information.
+**In VS Code:**
+- Coverage is automatically displayed in the file explorer (showing % next to Python files)
+- Coverage gutters appear in open Python files (green/red/orange lines)
+- Install the "Coverage Gutters" extension for enhanced visualization
+
+**In Browser:**
+- Open `htmlcov/index.html` in your browser for detailed coverage information
 
 ## Test Infrastructure
 

tests/Test-Matrix.md

@@ -2,14 +2,14 @@
 
 **Date / time**: __________________
 
-| Sample / Infrastructure     | SIMPLE APIM                 | APIM ACA                    | AFD APIM PE                 | App Gateway APIM ACA        |
-|:----------------------------|-----------------------------|-----------------------------|-----------------------------|-----------------------------|
-| **INFRASTRUCTURE**          | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container |
-| **authX**                   | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container |
-| **authX-pro**               | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container |
-| **azure-maps**              | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container |
-| **general**                 | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container |
-| **load-balancing**          | **N/A**                     | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container |
-| **oauth-3rd-party**         | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container |
-| **secure-blob-access**      | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container |
-| **INFRASTRUCTURE clean-up** | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container |
+| Sample / Infrastructure     | SIMPLE APIM                 | APIM ACA                    | AFD APIM PE                 | App Gateway APIM ACA        | App Gateway APIM PE         |
+|:----------------------------|-----------------------------|-----------------------------|-----------------------------|-----------------------------|-----------------------------|
+| **INFRASTRUCTURE**          | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container |
+| **authX**                   | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container |
+| **authX-pro**               | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container |
+| **azure-maps**              | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container |
+| **general**                 | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container |
+| **load-balancing**          | **N/A**                     | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container |
+| **oauth-3rd-party**         | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container |
+| **secure-blob-access**      | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container |
+| **INFRASTRUCTURE clean-up** | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container |

tests/python/.pylintrc

@@ -20,6 +20,7 @@ disable =
     R0801,  # Duplicate code
     R0902,  # Too many instance attributes
     R0903,  # Too few public methods
+    R0904,  # Too many public methods
     R0911,  # Too many return statements
     R0912,  # Too many branches
     R0913,  # Too many arguments

tests/python/check_python.ps1

@@ -77,9 +77,29 @@ Write-Host "  Step 2/2: Running Tests    " -ForegroundColor Yellow
 Write-Host "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" -ForegroundColor Yellow
 Write-Host ""
 
-& "$ScriptDir\run_tests.ps1"
+# Capture test output and pass it through to console while also capturing it
+$TestOutput = @()
+& "$ScriptDir\run_tests.ps1" 2>&1 | Tee-Object -Variable TestOutput | Write-Host
 $TestExitCode = $LASTEXITCODE
 
+# Parse test results from captured output
+$TotalTests = 0
+$PassedTests = 0
+$FailedTests = 0
+
+foreach ($Line in $TestOutput) {
+    $LineStr = $Line.ToString()
+    # Look for pytest summary line like "908 passed, 9 failed in 26.76s"
+    if ($LineStr -match '(\d+)\s+passed') {
+        $PassedTests = [int]::Parse($matches[1])
+    }
+    if ($LineStr -match '(\d+)\s+failed') {
+        $FailedTests = [int]::Parse($matches[1])
+    }
+}
+
+$TotalTests = $PassedTests + $FailedTests
+
 Write-Host ""
 
 
@@ -92,9 +112,11 @@ Write-Host "║         Final Results                      ║" -ForegroundColor
 Write-Host "╚════════════════════════════════════════════╝" -ForegroundColor Cyan
 Write-Host ""
 
+# Determine statuses
 $LintStatus = if ($LintExitCode -eq 0) { "✅ PASSED" } else { "⚠️  ISSUES FOUND" }
 $TestStatus = if ($TestExitCode -eq 0) { "✅ PASSED" } else { "❌ FAILED" }
 
+# Get pylint score
 $PylintScore = $null
 $LatestPylintText = Join-Path $ScriptDir "pylint/reports/latest.txt"
 if (Test-Path $LatestPylintText) {
@@ -104,17 +126,39 @@ if (Test-Path $LatestPylintText) {
     }
 }
 
-if ($PylintScore) {
-    $LintStatus = "$LintStatus ($PylintScore)"
-}
-
+# Set colors
 $LintColor = if ($LintExitCode -eq 0) { "Green" } else { "Yellow" }
 $TestColor = if ($TestExitCode -eq 0) { "Green" } else { "Red" }
 
-Write-Host "   Pylint:  " -NoNewline
-Write-Host $LintStatus -ForegroundColor $LintColor
-Write-Host "   Tests:   " -NoNewline
+# Calculate column widths for alignment
+$LabelWidth = "Pylint :".Length  # 7
+$Padding = " " * ($LabelWidth - 1)
+
+# Display Pylint status with score
+Write-Host "Pylint : " -NoNewline
+Write-Host $LintStatus -ForegroundColor $LintColor -NoNewline
+if ($PylintScore) {
+    Write-Host " ($PylintScore)" -ForegroundColor Gray
+} else {
+    Write-Host ""
+}
+
+# Display Test status with counts
+Write-Host "Tests  : " -NoNewline
 Write-Host $TestStatus -ForegroundColor $TestColor
+
+# Display test counts with right-aligned numbers
+if ($TotalTests -gt 0) {
+    # Calculate padding for right-alignment (max 5 digits)
+    $TotalPadded = "{0,5}" -f $TotalTests
+    $PassedPadded = "{0,5}" -f $PassedTests
+    $FailedPadded = "{0,5}" -f $FailedTests
+
+    Write-Host "          • Total  : $TotalPadded" -ForegroundColor Gray
+    Write-Host "          • Passed : $PassedPadded" -ForegroundColor Gray
+    Write-Host "          • Failed : $FailedPadded" -ForegroundColor Gray
+}
+
 Write-Host ""
 
 # Determine overall exit code
@@ -127,7 +171,7 @@ if ($TestExitCode -ne 0) {
 }
 
 if ($OverallExitCode -eq 0) {
-    Write-Host "🎉 All checks passed! Code is ready for commit." -ForegroundColor Green
+    Write-Host "🎉 All checks passed! Code is ready to commit." -ForegroundColor Green
 } else {
     Write-Host "⚠️  Some checks did not pass. Please review and fix issues." -ForegroundColor Yellow
 }

tests/python/check_python.sh

@@ -67,10 +67,18 @@ echo "━━━━━━━━━━━━━━━━━━━━━━━━
 echo ""
 
 set +e
-"$SCRIPT_DIR/run_tests.sh"
+TEST_OUTPUT=$("$SCRIPT_DIR/run_tests.sh" 2>&1)
 TEST_EXIT_CODE=$?
 set -e
 
+# Print the test output
+echo "$TEST_OUTPUT"
+
+# Parse test results from output
+PASSED_TESTS=$(echo "$TEST_OUTPUT" | grep -oE '[0-9]+ passed' | head -1 | grep -oE '[0-9]+' || echo "0")
+FAILED_TESTS=$(echo "$TEST_OUTPUT" | grep -oE '[0-9]+ failed' | head -1 | grep -oE '[0-9]+' || echo "0")
+TOTAL_TESTS=$((PASSED_TESTS + FAILED_TESTS))
+
 echo ""
 
 
@@ -83,24 +91,32 @@ echo "║         Final Results                                     ║"
 echo "╚═══════════════════════════════════════════════════════════╝"
 echo ""
 
+# Determine Pylint status
 if [ $LINT_EXIT_CODE -eq 0 ]; then
-    if [ -n "$PYLINT_SCORE" ]; then
-        echo "   Pylint:  ✅ PASSED ($PYLINT_SCORE)"
-    else
-        echo "   Pylint:  ✅ PASSED"
-    fi
+    LINT_STATUS="✅ PASSED"
 else
-    if [ -n "$PYLINT_SCORE" ]; then
-        echo "   Pylint:  ⚠️  ISSUES FOUND ($PYLINT_SCORE)"
-    else
-        echo "   Pylint:  ⚠️  ISSUES FOUND"
-    fi
+    LINT_STATUS="⚠️  ISSUES FOUND"
 fi
 
+# Determine Test status
 if [ $TEST_EXIT_CODE -eq 0 ]; then
-    echo "   Tests:   ✅ PASSED"
+    TEST_STATUS="✅ PASSED"
 else
-    echo "   Tests:   ❌ FAILED"
+    TEST_STATUS="❌ FAILED"
+fi
+
+# Display results with proper alignment
+echo "Pylint : $LINT_STATUS"
+if [ -n "$PYLINT_SCORE" ]; then
+    echo "         ($PYLINT_SCORE)"
+fi
+
+echo "Tests  : $TEST_STATUS"
+if [ $TOTAL_TESTS -gt 0 ]; then
+    # Right-align numbers with padding
+    printf "          • Total  : %5d\n" "$TOTAL_TESTS"
+    printf "          • Passed : %5d\n" "$PASSED_TESTS"
+    printf "          • Failed : %5d\n" "$FAILED_TESTS"
 fi
 
 echo ""
@@ -115,7 +131,7 @@ if [ $TEST_EXIT_CODE -ne 0 ]; then
 fi
 
 if [ $OVERALL_EXIT_CODE -eq 0 ]; then
-    echo "🎉 All checks passed! Code is ready for commit."
+    echo "🎉 All checks passed! Code is ready to commit."
 else
     echo "⚠️  Some checks did not pass. Please review and fix issues."
 fi

tests/python/conftest.py

@@ -10,6 +10,21 @@
 # Add the shared/python directory to the Python path for all tests
 sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '../../shared/python')))
 
+# Add the tests/python directory to import test_helpers
+sys.path.insert(0, os.path.abspath(os.path.dirname(__file__)))
+
+# APIM Samples imports
+# pylint: disable=wrong-import-position
+from test_helpers import (
+    create_mock_http_response,
+    create_mock_output,
+    create_sample_apis,
+    create_sample_policy_fragments,
+    get_sample_infrastructure_params,
+    MockApimRequestsPatches,
+    MockInfrastructuresPatches
+)
+
 
 # ------------------------------
 #    SHARED FIXTURES
@@ -34,3 +49,81 @@ def sample_test_data() -> dict[str, Any]:
         'test_resource_group': 'rg-test-apim-01',
         'test_location': 'eastus2'
     }
+
+
+# ------------------------------
+#    MOCK FIXTURES
+# ------------------------------
+
+@pytest.fixture(autouse=True)
+def infrastructures_patches():
+    """Automatically patch infrastructures dependencies for tests."""
+    with MockInfrastructuresPatches() as patches:
+        yield patches
+
+
+@pytest.fixture
+def mock_utils(infrastructures_patches):
+    """Return the patched utils module for infrastructures tests."""
+    return infrastructures_patches.utils
+
+
+@pytest.fixture
+def mock_az(infrastructures_patches):
+    """Return the patched azure_resources module for infrastructures tests."""
+    return infrastructures_patches.az
+
+
+@pytest.fixture
+def mock_az_success():
+    """Pre-configured successful Azure CLI output."""
+    return create_mock_output(success=True, json_data={'result': 'success'})
+
+
+@pytest.fixture
+def mock_az_failure():
+    """Pre-configured failed Azure CLI output."""
+    return create_mock_output(success=False, text='Error message')
+
+
+@pytest.fixture
+def sample_policy_fragments():
+    """Provide sample policy fragments for testing."""
+    return create_sample_policy_fragments(count=2)
+
+
+@pytest.fixture
+def sample_apis():
+    """Provide sample APIs for testing."""
+    return create_sample_apis(count=2)
+
+
+@pytest.fixture
+def sample_infrastructure_params() -> dict[str, Any]:
+    """Provide common infrastructure parameters."""
+    return get_sample_infrastructure_params()
+
+
+@pytest.fixture
+def mock_http_response_200():
+    """Pre-configured successful HTTP response."""
+    return create_mock_http_response(
+        status_code=200,
+        json_data={'result': 'ok'}
+    )
+
+
+@pytest.fixture
+def mock_http_response_error():
+    """Pre-configured error HTTP response."""
+    return create_mock_http_response(
+        status_code=500,
+        text='Internal Server Error'
+    )
+
+
+@pytest.fixture
+def apimrequests_patches():
+    """Provide common apimrequests patches for HTTP tests."""
+    with MockApimRequestsPatches() as patches:
+        yield patches