Skip to content

Commit b18bb4c

Browse files
attharva-24attharva-24
authored
fix(auth): parse hostname for mTLS and PSC endpoint certificate rotation (#18147) (#18201)
Relands #18153 with all review feedback addressed, full test coverage, and live manual verification against Google Cloud mTLS endpoints. Fixes #18147 ### Changes * Replaced naive `any(prefix in url for prefix in MTLS_URL_PREFIXES)` substring matching with `_mtls_helper.is_mtls_endpoint(url)`. * Parsed hostname using `urllib.parse.urlsplit` to avoid false positives on URL path/query parameters. * Added support for Private Service Connect (PSC) custom domains (`*.p.googleapis.com`, `p.googleapis.com`). * Normalized hostnames by stripping trailing root dots (`.rstrip(".")`) for FQDNs. * Annotated `is_mtls_endpoint` with `object` instead of `Any` for strict static type checking. * Added comprehensive parametrized unit tests (57 cases in `test__mtls_helper.py`). --- ### Manual Verification & Findings 1. **Non-mTLS URLs (`storage.googleapis.com`)**: On receiving `401 Unauthorized`, `AuthorizedSession` executes the standard OAuth token refresh & retry (2 HTTP requests: 401 ➔ 200 OK), and completely **skips certificate rotation (`spy_check Call Count: 0`)**. 2. **PSC / mTLS URLs (`storage.p.googleapis.com`)**: On receiving `401 Unauthorized`, `AuthorizedSession` **triggers certificate rotation inspection (`spy_check Call Count: 1`)** before refreshing credentials and retrying with `200 OK`. <details> <summary><b>Click to expand Live Manual Test Output (100% Green)</b></summary> ```text ======================================================================== Real Production mTLS Certificate & Endpoint Live Manual Verification ======================================================================== =========================================================================== >>> 1. INSPECTING ACTUAL CLIENT CERTIFICATE ON MACHINE =========================================================================== Client Certificate Status: LOADED FROM DISK / ECP Certificate Size: 1428 bytes Private Key Size: 1675 bytes Certificate Subject: <Name(CN=www.example.com,OU=Google Testing unit,O=Google Testing)> Certificate Issuer: <Name(CN=Google Testing Intermediate CA,OU=Google Testing unit,O=Google Testing)> SHA-256 Fingerprint: <SHA256_CERT_FINGERPRINT_REDACTED> =========================================================================== >>> 2. ESTABLISHING REAL LIVE mTLS HANDSHAKE (pubsub.mtls.googleapis.com) =========================================================================== Session mTLS Enabled: True Cached Cert Size in Session: 1428 bytes Connecting to: https://pubsub.mtls.googleapis.com/v1/projects/<PROJECT_ID>/topics Live mTLS Handshake Status: SUCCESS (HTTP 403) =========================================================================== >>> 3. LIVE 401 TEST: NON-mTLS URL WITH PATH TRAP (Bug 1 Fix) =========================================================================== Target URL: https://storage.googleapis.com/storage/v1/b/mtls.googleapis.com Notice: Path has 'mtls.googleapis.com', but host is standard 'storage.googleapis.com' HTTP Requests Sent: 2 (1st: 401 Unauthorized -> 2nd [Retry]: 200 OK) Endpoint Detected as mTLS: False (Expected: False) spy_check Call Count: 0 (Expected: 0) [PASS] Successfully avoided cert rotation check (0 calls) despite 401 and retry! =========================================================================== >>> 4. LIVE 401 TEST: PRIVATE SERVICE CONNECT (PSC) URL (Bug 2 Fix) =========================================================================== Target URL: https://storage.p.googleapis.com/b/production-bucket Notice: Private Service Connect domain (*.p.googleapis.com) HTTP Requests Sent: 2 (1st: 401 Unauthorized -> 2nd [Retry]: 200 OK) Endpoint Detected as mTLS: True (Expected: True) spy_check Call Count: 1 (Expected: 1 on 401) Cert Check Triggered: True (Inspected disk certificate and computed fingerprint) [PASS] Successfully triggered cert inspection on 401 (1 call) and retried with 200 OK! ======================================================================== ALL REAL CERTIFICATE LIVE MANUAL TESTS PASSED SUCCESSFULLY! (100% GREEN) ========================================================================
1 parent ad2ef26 commit b18bb4c

6 files changed

Lines changed: 238 additions & 21 deletions

File tree

‎packages/google-auth/google/auth/transport/_mtls_helper.py‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@
2424
import sys
2525
import tempfile
2626
from typing import cast, Generator, List, Optional, Tuple, Union
27+
from urllib.parse import urlsplit
2728

2829
from google.auth import _agent_identity_utils
2930
from google.auth import _cloud_sdk
@@ -840,3 +841,50 @@ def call_client_cert_callback():
840841
generate_encrypted_key=True
841842
)
842843
return cert_bytes, key_bytes
844+
845+
846+
_MTLS_HOST_SUFFIXES = (
847+
".mtls.googleapis.com",
848+
".mtls.sandbox.googleapis.com",
849+
".p.googleapis.com",
850+
)
851+
_MTLS_EXACT_HOSTS = (
852+
"mtls.googleapis.com",
853+
"mtls.sandbox.googleapis.com",
854+
"p.googleapis.com",
855+
)
856+
857+
858+
def is_mtls_endpoint(url: Optional[Union[str, bytes, object]]) -> bool:
859+
"""Checks if the given URL corresponds to an mTLS or Private Service Connect (PSC) endpoint.
860+
861+
Args:
862+
url (Optional[Union[str, bytes, object]]): The request URL.
863+
864+
Returns:
865+
bool: True if the URL targets an mTLS or PSC endpoint, False otherwise.
866+
"""
867+
if not url:
868+
return False
869+
if hasattr(url, "url") and isinstance(url.url, (str, bytes)):
870+
url = url.url
871+
if isinstance(url, bytes):
872+
try:
873+
url = url.decode("utf-8")
874+
except (UnicodeDecodeError, AttributeError):
875+
return False
876+
elif not isinstance(url, str):
877+
url = str(url)
878+
try:
879+
hostname = urlsplit(url).hostname
880+
except (ValueError, TypeError, AttributeError):
881+
return False
882+
883+
if not hostname:
884+
return False
885+
886+
hostname = hostname.rstrip(".").lower()
887+
if not hostname:
888+
return False
889+
890+
return hostname in _MTLS_EXACT_HOSTS or hostname.endswith(_MTLS_HOST_SUFFIXES)

‎packages/google-auth/google/auth/transport/requests.py‎

Lines changed: 1 addition & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -647,13 +647,7 @@ def request(
647647
):
648648
# Handle unauthorized permission error(401 status code)
649649
if response.status_code == http_client.UNAUTHORIZED:
650-
MTLS_URL_PREFIXES = [
651-
"mtls.googleapis.com",
652-
"mtls.sandbox.googleapis.com",
653-
]
654-
use_mtls = self.is_mtls and any(
655-
prefix in url for prefix in MTLS_URL_PREFIXES
656-
)
650+
use_mtls = self.is_mtls and _mtls_helper.is_mtls_endpoint(url)
657651
if use_mtls:
658652
(
659653
call_cert_bytes,

‎packages/google-auth/google/auth/transport/urllib3.py‎

Lines changed: 1 addition & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -409,11 +409,6 @@ def urlopen(self, method, url, body=None, headers=None, **kwargs):
409409
if headers is None:
410410
headers = self.headers
411411

412-
use_mtls = False
413-
if self._is_mtls:
414-
MTLS_URL_PREFIXES = ["mtls.googleapis.com", "mtls.sandbox.googleapis.com"]
415-
use_mtls = any([prefix in url for prefix in MTLS_URL_PREFIXES])
416-
417412
# Make a copy of the headers. They will be modified by the credentials
418413
# and we want to pass the original headers if we recurse.
419414
request_headers = headers.copy()
@@ -436,6 +431,7 @@ def urlopen(self, method, url, body=None, headers=None, **kwargs):
436431
and _credential_refresh_attempt < self._max_refresh_attempts
437432
):
438433
if response.status == http_client.UNAUTHORIZED:
434+
use_mtls = self._is_mtls and _mtls_helper.is_mtls_endpoint(url)
439435
if use_mtls:
440436
(
441437
call_cert_bytes,

‎packages/google-auth/tests/transport/test__mtls_helper.py‎

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@
2121
from cryptography.hazmat.primitives import hashes, serialization
2222
from cryptography.hazmat.primitives.asymmetric import ec
2323
import pytest # type: ignore
24+
import urllib3.util
2425

2526
from google.auth import environment_vars, exceptions
2627
from google.auth.transport import _mtls_helper
@@ -1888,3 +1889,80 @@ def test_remove_oserror_ignored(
18881889
mock_fh.flush.assert_called_once()
18891890
mock_fsync.assert_called_once()
18901891
mock_remove.assert_called_once_with("/path/to/secret")
1892+
1893+
1894+
class TestIsMtlsEndpoint(object):
1895+
@pytest.mark.parametrize(
1896+
"url",
1897+
[
1898+
"https://mtls.googleapis.com",
1899+
"https://mtls.googleapis.com/",
1900+
"https://mtls.googleapis.com/v1/projects",
1901+
"https://mtls.sandbox.googleapis.com",
1902+
"https://mtls.sandbox.googleapis.com/v1/projects",
1903+
"https://pubsub.mtls.googleapis.com",
1904+
"https://pubsub.mtls.googleapis.com/v1/projects/my-project",
1905+
"https://storage.mtls.sandbox.googleapis.com/b/my-bucket",
1906+
"https://my-service.us-east1.rep.mtls.googleapis.com/v1",
1907+
"https://my-service.us-east1.rep.mtls.sandbox.googleapis.com/v1",
1908+
"https://storage.p.googleapis.com/b/my-bucket",
1909+
"https://my-custom-endpoint.p.googleapis.com/v1",
1910+
"https://my-service.us-east1.p.googleapis.com/v1",
1911+
"HTTP://PUBSUB.MTLS.GOOGLEAPIS.COM/V1",
1912+
b"https://pubsub.mtls.googleapis.com",
1913+
b"https://storage.p.googleapis.com/b/my-bucket",
1914+
urllib3.util.parse_url("https://pubsub.mtls.googleapis.com/v1"),
1915+
urllib3.util.parse_url("https://storage.p.googleapis.com/b/my-bucket"),
1916+
"https://pubsub.mtls.googleapis.com.",
1917+
"https://storage.p.googleapis.com./b/my-bucket",
1918+
"https://mtls.googleapis.com.",
1919+
"https://pubsub.mtls.googleapis.com:443/v1",
1920+
"https://pubsub.mtls.googleapis.com:8443/v1",
1921+
"https://storage.p.googleapis.com:443/b/my-bucket",
1922+
"https://pubsub.mtls.googleapis.com/v1/projects?pageSize=10#frag",
1923+
"https://pubsub.mtls.googleapis.com:443/v1/projects?pageSize=10&filter=foo#frag",
1924+
"https://storage.p.googleapis.com:443/b/my-bucket?param=1#section",
1925+
"https://mtls.googleapis.com:443/",
1926+
"https://p.googleapis.com",
1927+
"https://p.googleapis.com/",
1928+
"https://p.googleapis.com:443/v1",
1929+
"https://p.googleapis.com.",
1930+
],
1931+
)
1932+
def test_is_mtls_endpoint_true(self, url):
1933+
assert _mtls_helper.is_mtls_endpoint(url) is True
1934+
1935+
@pytest.mark.parametrize(
1936+
"url",
1937+
[
1938+
"https://storage.googleapis.com",
1939+
"https://storage.googleapis.com.",
1940+
"https://storage.googleapis.com:443/b/my-bucket",
1941+
"https://storage.googleapis.com:443/bucket/mtls.googleapis.com?pageSize=10#frag",
1942+
"https://storage.googleapis.com/bucket/mtls.googleapis.com",
1943+
"https://[2001:db8::1]:443/mtls.googleapis.com",
1944+
"https://[::1]:8443/mtls.googleapis.com",
1945+
"https://logging.googleapis.com/v2/entries?filter=mtls.googleapis.com",
1946+
"https://logging.googleapis.com/v2/entries?filter=mtls.sandbox.googleapis.com",
1947+
"https://logging.googleapis.com/v2/entries?filter=service.p.googleapis.com",
1948+
"https://example.com/mtls.googleapis.com",
1949+
"https://fake-mtls.googleapis.com.attacker.com/v1",
1950+
"https://fake-p.googleapis.com.attacker.com/v1",
1951+
"http://localhost:8080/",
1952+
"http://localhost:8080/mtls.googleapis.com",
1953+
b"https://storage.googleapis.com",
1954+
b"https://storage.googleapis.com/bucket/mtls.googleapis.com",
1955+
b"\xff\xfeinvalid",
1956+
urllib3.util.parse_url("https://storage.googleapis.com/b/my-bucket"),
1957+
urllib3.util.parse_url(
1958+
"https://storage.googleapis.com/bucket/mtls.googleapis.com"
1959+
),
1960+
"https://.",
1961+
"",
1962+
None,
1963+
123,
1964+
"not a url",
1965+
],
1966+
)
1967+
def test_is_mtls_endpoint_false(self, url):
1968+
assert _mtls_helper.is_mtls_endpoint(url) is False

‎packages/google-auth/tests/transport/test_requests.py‎

Lines changed: 45 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -664,6 +664,9 @@ def test_configure_mtls_channel_cert_loading_exceptions(
664664

665665
assert not auth_session.is_mtls
666666

667+
@mock.patch(
668+
"google.auth.transport._mtls_helper._get_cert_config_path", return_value=None
669+
)
667670
@mock.patch(
668671
"google.auth.transport._mtls_helper.get_client_cert_and_key", autospec=True
669672
)
@@ -677,7 +680,7 @@ def test_configure_mtls_channel_cert_loading_exceptions(
677680
},
678681
)
679682
def test_configure_mtls_channel_without_client_cert_env(
680-
self, get_client_cert_and_key
683+
self, get_client_cert_and_key, mock_get_cert_config_path
681684
):
682685
env_to_patch = {
683686
environment_vars.GOOGLE_API_USE_CLIENT_CERTIFICATE: "",
@@ -939,7 +942,7 @@ def test_cert_rotation_logic_skipped_on_other_refresh_status_codes(self):
939942

940943
def test_cert_rotation_skipped_on_non_mtls_url(self):
941944
"""
942-
Tests that mTLS cert rotation is skipped on a non-mTLS URL even if
945+
Tests that mTLS cert rotation is skipped on non-mTLS URLs even if
943946
mTLS is enabled and an UNAUTHORIZED (401) response is received.
944947
"""
945948
credentials = mock.Mock(wraps=CredentialsStub())
@@ -950,22 +953,56 @@ def test_cert_rotation_skipped_on_non_mtls_url(self):
950953
make_response(status=http_client.OK),
951954
]
952955
)
956+
non_mtls_url = "https://storage.googleapis.com/bucket/mtls.googleapis.com"
953957
authed_session = google.auth.transport.requests.AuthorizedSession(
954958
credentials, refresh_timeout=60
955959
)
956-
authed_session.mount(self.TEST_URL, adapter)
960+
authed_session.mount("https://", adapter)
957961
authed_session._is_mtls = True
962+
authed_session._cached_cert = b"cached_cert"
958963

959-
with mock.patch(
960-
"google.auth.transport.requests._mtls_helper", autospec=True
961-
) as mock_helper:
962-
authed_session.request("GET", self.TEST_URL)
964+
with mock.patch.object(
965+
google.auth.transport._mtls_helper,
966+
"check_parameters_for_unauthorized_response",
967+
) as mock_check_params:
968+
authed_session.request("GET", non_mtls_url)
963969

964970
# Assert refresh happened
965971
assert credentials.refresh.called
966972

967973
# Assert mTLS check logic was SKIPPED
968-
assert not mock_helper.check_parameters_for_unauthorized_response.called
974+
assert not mock_check_params.called
975+
976+
def test_cert_rotation_triggered_on_psc_url(self):
977+
"""
978+
Tests that mTLS cert rotation IS triggered on a Private Service Connect
979+
(PSC) mTLS endpoint when an UNAUTHORIZED (401) response is received.
980+
"""
981+
credentials = mock.Mock(wraps=CredentialsStub())
982+
adapter = AdapterStub(
983+
[
984+
make_response(status=http_client.UNAUTHORIZED),
985+
make_response(status=http_client.OK),
986+
]
987+
)
988+
psc_url = "https://storage.p.googleapis.com/b/my-bucket"
989+
authed_session = google.auth.transport.requests.AuthorizedSession(
990+
credentials, refresh_timeout=60
991+
)
992+
authed_session.mount(psc_url, adapter)
993+
authed_session._is_mtls = True
994+
authed_session._cached_cert = b"cached_cert"
995+
996+
with mock.patch.object(
997+
google.auth.transport._mtls_helper,
998+
"check_parameters_for_unauthorized_response",
999+
return_value=(b"new_cert", b"new_key", "old_fp", "old_fp"),
1000+
) as mock_check_params:
1001+
authed_session.request("GET", psc_url)
1002+
1003+
# Assert mTLS check logic was called on PSC endpoint
1004+
mock_check_params.assert_called_once()
1005+
assert credentials.refresh.called
9691006

9701007
def test_configure_mtls_channel_subsequent_failure(self):
9711008
# 1. Setup successful mTLS configuration

‎packages/google-auth/tests/transport/test_urllib3.py‎

Lines changed: 65 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -385,6 +385,9 @@ def test_configure_mtls_channel_cert_loading_exceptions(
385385

386386
assert not authed_http._is_mtls
387387

388+
@mock.patch(
389+
"google.auth.transport._mtls_helper._get_cert_config_path", return_value=None
390+
)
388391
@mock.patch(
389392
"google.auth.transport._mtls_helper.get_client_cert_and_key", autospec=True
390393
)
@@ -398,7 +401,7 @@ def test_configure_mtls_channel_cert_loading_exceptions(
398401
},
399402
)
400403
def test_configure_mtls_channel_without_client_cert_env(
401-
self, get_client_cert_and_key
404+
self, get_client_cert_and_key, mock_get_cert_config_path
402405
):
403406
callback = mock.Mock()
404407

@@ -655,6 +658,67 @@ def test_cert_rotation_logic_skipped_on_other_refresh_status_codes(self):
655658
# Assert mTLS check logic was SKIPPED (Inner Check was False)
656659
assert not mock_helper.check_parameters_for_unauthorized_response.called
657660

661+
def test_cert_rotation_skipped_on_non_mtls_url(self):
662+
"""
663+
Tests that mTLS cert rotation is skipped on non-mTLS URLs even if
664+
mTLS is enabled and an UNAUTHORIZED (401) response is received.
665+
"""
666+
credentials = mock.Mock(wraps=CredentialsStub())
667+
http = HttpStub(
668+
[
669+
ResponseStub(status=http_client.UNAUTHORIZED),
670+
ResponseStub(status=http_client.OK),
671+
]
672+
)
673+
non_mtls_url = "https://storage.googleapis.com/bucket/mtls.googleapis.com"
674+
authed_http = google.auth.transport.urllib3.AuthorizedHttp(
675+
credentials, http=http
676+
)
677+
authed_http._is_mtls = True
678+
authed_http._cached_cert = b"cached_cert"
679+
680+
with mock.patch.object(
681+
google.auth.transport._mtls_helper,
682+
"check_parameters_for_unauthorized_response",
683+
) as mock_check_params:
684+
authed_http.urlopen("GET", non_mtls_url)
685+
686+
# Assert refresh happened
687+
assert credentials.refresh.called
688+
689+
# Assert mTLS check logic was SKIPPED
690+
assert not mock_check_params.called
691+
692+
def test_cert_rotation_triggered_on_psc_url(self):
693+
"""
694+
Tests that mTLS cert rotation IS triggered on a Private Service Connect
695+
(PSC) mTLS endpoint when an UNAUTHORIZED (401) response is received.
696+
"""
697+
credentials = mock.Mock(wraps=CredentialsStub())
698+
http = HttpStub(
699+
[
700+
ResponseStub(status=http_client.UNAUTHORIZED),
701+
ResponseStub(status=http_client.OK),
702+
]
703+
)
704+
psc_url = "https://storage.p.googleapis.com/b/my-bucket"
705+
authed_http = google.auth.transport.urllib3.AuthorizedHttp(
706+
credentials, http=http
707+
)
708+
authed_http._is_mtls = True
709+
authed_http._cached_cert = b"cached_cert"
710+
711+
with mock.patch.object(
712+
google.auth.transport._mtls_helper,
713+
"check_parameters_for_unauthorized_response",
714+
return_value=(b"new_cert", b"new_key", "old_fp", "old_fp"),
715+
) as mock_check_params:
716+
authed_http.urlopen("GET", psc_url)
717+
718+
# Assert mTLS check logic was called on PSC endpoint
719+
mock_check_params.assert_called_once()
720+
assert credentials.refresh.called
721+
658722
@mock.patch("google.auth.transport.urllib3._make_mutual_tls_http", autospec=True)
659723
def test_configure_mtls_channel_subsequent_failure(self, mock_make_mutual_tls_http):
660724
callback = mock.Mock()

0 commit comments

Comments
 (0)