Skip to content

Commit f53ec37

Browse files
Python Test Refactoring (#115)
1 parent cd6a755 commit f53ec37

24 files changed

+4637
-2247
lines changed

.github/python.instructions.md

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -48,14 +48,21 @@ applyTo: '**/*.py'
4848
2. Specific type/constant imports (e.g., `from apimtypes import INFRASTRUCTURE`)
4949
3. Specific function imports (e.g., `from console import print_error`)
5050

51-
## Linting (pylint)
51+
## Code Quality Checklist
5252

53-
- Respect the repository pylint configuration at `tests/python/.pylintrc`.
54-
- When changing Python code, run pylint and ensure changes do not worsen the pylint rating unexpectedly.
55-
- Prefer fixing root causes (e.g., import structure, error handling) over suppressions.
53+
Before completing any Python code changes, verify:
54+
55+
- [ ] All pylint warnings and errors are resolved (`pylint --rcfile=tests/python/.pylintrc <file>`)
56+
- [ ] Code follows PEP 8 and the style guidelines in this file
57+
- [ ] Import statements for modules within this repo are placed last in the imports and are grouped with the `# APIM Samples imports` header
58+
- [ ] Type hints are present where appropriate
59+
- [ ] No unnecessary comments; docstrings are present for functions and classes
60+
- [ ] Edge cases and error handling are implemented
61+
- [ ] Prefer fixing root causes (e.g., import structure, error handling) over suppressions.
5662

5763
## Testing
5864

65+
- Aim for 90+% code coverage for each file.
5966
- Add or update pytest unit tests when changing behavior.
6067
- Prefer focused tests for the code being changed.
6168
- Avoid tests that require live Azure access; mock Azure CLI interactions and `azure_resources` helpers.

.gitignore

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,10 @@ labs-in-progress/
2626

2727
# Coverage data and reports
2828
.coverage
29+
.coverage.*
30+
coverage.xml
31+
coverage.json
32+
htmlcov/
2933
tests/python/htmlcov/
3034

3135
# Pylint reports
@@ -40,3 +44,5 @@ Test-Matrix.html
4044

4145
$JsonReport
4246
$TextReport
47+
$JsonReportRelative
48+
$TextReportRelative

shared/python/infrastructures.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -145,7 +145,7 @@ def _create_keyvault(self, key_vault_name: str) -> bool:
145145
print_error('Failed to assign Key Vault Certificates Officer role to current user.\nThis is an RBAC permission issue - verify your account has sufficient permissions.')
146146
return False
147147

148-
print_ok(' Assigned Key Vault Certificates Officer role to current user')
148+
print_ok('Assigned Key Vault Certificates Officer role to current user')
149149

150150
# Brief wait for role assignment propagation
151151
print_plain('⏳ Waiting for role assignment propagation (15 seconds)...')

shared/python/utils.py

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -117,7 +117,6 @@ def __init__(self, rg_location: str, deployment: INFRASTRUCTURE, index: int, api
117117
print_val('Infrastructure', self.deployment.value)
118118
print_val('Index', self.index)
119119
print_val('APIM SKU', self.apim_sku.value)
120-
print_plain('')
121120

122121
# ------------------------------
123122
# PUBLIC METHODS

tests/README.md

Lines changed: 18 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -80,12 +80,27 @@ Run tests separately when you only need test execution:
8080

8181
Both scripts:
8282
- Run all tests in `tests/python` using pytest
83-
- Generate a code coverage report (HTML output in `tests/python/htmlcov`)
84-
- Store the raw coverage data in `tests/python/.coverage`
83+
- Generate code coverage reports:
84+
- HTML: `htmlcov/index.html` (at repository root)
85+
- XML: `coverage.xml` (for VS Code integration)
86+
- JSON: `coverage.json`
87+
- Store the raw coverage data in `.coverage` (at repository root)
8588

8689
#### Viewing Coverage Reports
8790

88-
After running tests, open `tests/python/htmlcov/index.html` in your browser to view detailed coverage information.
91+
**In VS Code (no extra extensions):**
92+
- Open the Testing view (beaker icon in the Activity Bar).
93+
- Click the "Toggle Code Coverage" shield button in the Testing toolbar.
94+
- Run tests from the Testing view (Run All or individual test runs).
95+
- The Explorer will decorate Python files with coverage percentages, and the editor will show covered/uncovered lines.
96+
- Make sure the Python extension is enabled and `coverage`/`pytest-cov` are available in your venv. If needed:
97+
```powershell
98+
pip install coverage pytest-cov
99+
```
100+
- Note: Running pytest only from the terminal won’t decorate the Explorer. Use the Testing UI to see coverage overlays.
101+
102+
**In Browser:**
103+
- Open `htmlcov/index.html` in your browser for detailed coverage information
89104

90105
## Test Infrastructure
91106

tests/Test-Matrix.md

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -2,14 +2,14 @@
22

33
**Date / time**: __________________
44

5-
| Sample / Infrastructure | SIMPLE APIM | APIM ACA | AFD APIM PE | App Gateway APIM ACA |
6-
|:----------------------------|-----------------------------|-----------------------------|-----------------------------|-----------------------------|
7-
| **INFRASTRUCTURE** | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container |
8-
| **authX** | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container |
9-
| **authX-pro** | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container |
10-
| **azure-maps** | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container |
11-
| **general** | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container |
12-
| **load-balancing** | **N/A** | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container |
13-
| **oauth-3rd-party** | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container |
14-
| **secure-blob-access** | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container |
15-
| **INFRASTRUCTURE clean-up** | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container |
5+
| Sample / Infrastructure | SIMPLE APIM | APIM ACA | AFD APIM PE | App Gateway APIM ACA | App Gateway APIM PE |
6+
|:----------------------------|-----------------------------|-----------------------------|-----------------------------|-----------------------------|-----------------------------|
7+
| **INFRASTRUCTURE** | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container |
8+
| **authX** | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container |
9+
| **authX-pro** | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container |
10+
| **azure-maps** | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container |
11+
| **general** | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container |
12+
| **load-balancing** | **N/A** | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container |
13+
| **oauth-3rd-party** | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container |
14+
| **secure-blob-access** | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container |
15+
| **INFRASTRUCTURE clean-up** | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container | ▢ Local<br>▢ Dev Container |

tests/python/.pylintrc

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@ disable =
2020
R0801, # Duplicate code
2121
R0902, # Too many instance attributes
2222
R0903, # Too few public methods
23+
R0904, # Too many public methods
2324
R0911, # Too many return statements
2425
R0912, # Too many branches
2526
R0913, # Too many arguments

tests/python/check_python.ps1

Lines changed: 53 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -77,9 +77,29 @@ Write-Host " Step 2/2: Running Tests " -ForegroundColor Yellow
7777
Write-Host "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" -ForegroundColor Yellow
7878
Write-Host ""
7979

80-
& "$ScriptDir\run_tests.ps1"
80+
# Capture test output and pass it through to console while also capturing it
81+
$TestOutput = @()
82+
& "$ScriptDir\run_tests.ps1" 2>&1 | Tee-Object -Variable TestOutput | Write-Host
8183
$TestExitCode = $LASTEXITCODE
8284

85+
# Parse test results from captured output
86+
$TotalTests = 0
87+
$PassedTests = 0
88+
$FailedTests = 0
89+
90+
foreach ($Line in $TestOutput) {
91+
$LineStr = $Line.ToString()
92+
# Look for pytest summary line like "908 passed, 9 failed in 26.76s"
93+
if ($LineStr -match '(\d+)\s+passed') {
94+
$PassedTests = [int]::Parse($matches[1])
95+
}
96+
if ($LineStr -match '(\d+)\s+failed') {
97+
$FailedTests = [int]::Parse($matches[1])
98+
}
99+
}
100+
101+
$TotalTests = $PassedTests + $FailedTests
102+
83103
Write-Host ""
84104

85105

@@ -92,9 +112,11 @@ Write-Host "║ Final Results ║" -ForegroundColor
92112
Write-Host "╚════════════════════════════════════════════╝" -ForegroundColor Cyan
93113
Write-Host ""
94114

115+
# Determine statuses
95116
$LintStatus = if ($LintExitCode -eq 0) { "✅ PASSED" } else { "⚠️ ISSUES FOUND" }
96117
$TestStatus = if ($TestExitCode -eq 0) { "✅ PASSED" } else { "❌ FAILED" }
97118

119+
# Get pylint score
98120
$PylintScore = $null
99121
$LatestPylintText = Join-Path $ScriptDir "pylint/reports/latest.txt"
100122
if (Test-Path $LatestPylintText) {
@@ -104,17 +126,39 @@ if (Test-Path $LatestPylintText) {
104126
}
105127
}
106128

107-
if ($PylintScore) {
108-
$LintStatus = "$LintStatus ($PylintScore)"
109-
}
110-
129+
# Set colors
111130
$LintColor = if ($LintExitCode -eq 0) { "Green" } else { "Yellow" }
112131
$TestColor = if ($TestExitCode -eq 0) { "Green" } else { "Red" }
113132

114-
Write-Host " Pylint: " -NoNewline
115-
Write-Host $LintStatus -ForegroundColor $LintColor
116-
Write-Host " Tests: " -NoNewline
133+
# Calculate column widths for alignment
134+
$LabelWidth = "Pylint :".Length # 7
135+
$Padding = " " * ($LabelWidth - 1)
136+
137+
# Display Pylint status with score
138+
Write-Host "Pylint : " -NoNewline
139+
Write-Host $LintStatus -ForegroundColor $LintColor -NoNewline
140+
if ($PylintScore) {
141+
Write-Host " ($PylintScore)" -ForegroundColor Gray
142+
} else {
143+
Write-Host ""
144+
}
145+
146+
# Display Test status with counts
147+
Write-Host "Tests : " -NoNewline
117148
Write-Host $TestStatus -ForegroundColor $TestColor
149+
150+
# Display test counts with right-aligned numbers
151+
if ($TotalTests -gt 0) {
152+
# Calculate padding for right-alignment (max 5 digits)
153+
$TotalPadded = "{0,5}" -f $TotalTests
154+
$PassedPadded = "{0,5}" -f $PassedTests
155+
$FailedPadded = "{0,5}" -f $FailedTests
156+
157+
Write-Host " • Total : $TotalPadded" -ForegroundColor Gray
158+
Write-Host " • Passed : $PassedPadded" -ForegroundColor Gray
159+
Write-Host " • Failed : $FailedPadded" -ForegroundColor Gray
160+
}
161+
118162
Write-Host ""
119163

120164
# Determine overall exit code
@@ -127,7 +171,7 @@ if ($TestExitCode -ne 0) {
127171
}
128172

129173
if ($OverallExitCode -eq 0) {
130-
Write-Host "🎉 All checks passed! Code is ready for commit." -ForegroundColor Green
174+
Write-Host "🎉 All checks passed! Code is ready to commit." -ForegroundColor Green
131175
} else {
132176
Write-Host "⚠️ Some checks did not pass. Please review and fix issues." -ForegroundColor Yellow
133177
}

tests/python/check_python.sh

Lines changed: 30 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -67,10 +67,18 @@ echo "━━━━━━━━━━━━━━━━━━━━━━━━
6767
echo ""
6868

6969
set +e
70-
"$SCRIPT_DIR/run_tests.sh"
70+
TEST_OUTPUT=$("$SCRIPT_DIR/run_tests.sh" 2>&1)
7171
TEST_EXIT_CODE=$?
7272
set -e
7373

74+
# Print the test output
75+
echo "$TEST_OUTPUT"
76+
77+
# Parse test results from output
78+
PASSED_TESTS=$(echo "$TEST_OUTPUT" | grep -oE '[0-9]+ passed' | head -1 | grep -oE '[0-9]+' || echo "0")
79+
FAILED_TESTS=$(echo "$TEST_OUTPUT" | grep -oE '[0-9]+ failed' | head -1 | grep -oE '[0-9]+' || echo "0")
80+
TOTAL_TESTS=$((PASSED_TESTS + FAILED_TESTS))
81+
7482
echo ""
7583

7684

@@ -83,24 +91,32 @@ echo "║ Final Results ║"
8391
echo "╚═══════════════════════════════════════════════════════════╝"
8492
echo ""
8593

94+
# Determine Pylint status
8695
if [ $LINT_EXIT_CODE -eq 0 ]; then
87-
if [ -n "$PYLINT_SCORE" ]; then
88-
echo " Pylint: ✅ PASSED ($PYLINT_SCORE)"
89-
else
90-
echo " Pylint: ✅ PASSED"
91-
fi
96+
LINT_STATUS="✅ PASSED"
9297
else
93-
if [ -n "$PYLINT_SCORE" ]; then
94-
echo " Pylint: ⚠️ ISSUES FOUND ($PYLINT_SCORE)"
95-
else
96-
echo " Pylint: ⚠️ ISSUES FOUND"
97-
fi
98+
LINT_STATUS="⚠️ ISSUES FOUND"
9899
fi
99100

101+
# Determine Test status
100102
if [ $TEST_EXIT_CODE -eq 0 ]; then
101-
echo " Tests: ✅ PASSED"
103+
TEST_STATUS="✅ PASSED"
102104
else
103-
echo " Tests: ❌ FAILED"
105+
TEST_STATUS="❌ FAILED"
106+
fi
107+
108+
# Display results with proper alignment
109+
echo "Pylint : $LINT_STATUS"
110+
if [ -n "$PYLINT_SCORE" ]; then
111+
echo " ($PYLINT_SCORE)"
112+
fi
113+
114+
echo "Tests : $TEST_STATUS"
115+
if [ $TOTAL_TESTS -gt 0 ]; then
116+
# Right-align numbers with padding
117+
printf " • Total : %5d\n" "$TOTAL_TESTS"
118+
printf " • Passed : %5d\n" "$PASSED_TESTS"
119+
printf " • Failed : %5d\n" "$FAILED_TESTS"
104120
fi
105121

106122
echo ""
@@ -115,7 +131,7 @@ if [ $TEST_EXIT_CODE -ne 0 ]; then
115131
fi
116132

117133
if [ $OVERALL_EXIT_CODE -eq 0 ]; then
118-
echo "🎉 All checks passed! Code is ready for commit."
134+
echo "🎉 All checks passed! Code is ready to commit."
119135
else
120136
echo "⚠️ Some checks did not pass. Please review and fix issues."
121137
fi

0 commit comments

Comments
 (0)