The latest azure.mgmt.rdbms returns publicNetworkAccess = Enabled whereas latest az cli returns publicNetworkAccess = Disabled #36459

antonkazlouski-orca · 2024-07-12T11:26:47Z

Package Name: azure-mgmt-rdbms
Package Version: 10.1.0
Operating System: Debian GNU/Linux 12 (bookworm)
Python Version: 3.11.4

Describe the bug
list_by_resource_group command of the latest azure.mgmt.rdbms returns publicNetworkAccess = Enabled whereas latest az cli returns publicNetworkAccess = Disabled.

To Reproduce
Steps to reproduce the behavior:

Create a Resource Group: Use the Azure CLI to create a resource group.
Create a MySQL Flexible Server: Use the Azure CLI to create a MySQL Flexible Server within the resource group.
Set Up Azure AD Application and Service Principal:
Create an Azure AD application.
Create a service principal for the application.
Assign the Contributor role to the service principal for the subscription.

import logging
from azure.identity import ClientSecretCredential
from azure.mgmt.rdbms.mysql_flexibleservers import MySQLManagementClient

logger = logging.getLogger('azure')
logger.setLevel(logging.DEBUG)

ch = logging.StreamHandler()
ch.setLevel(logging.DEBUG)
logger.addHandler(ch)

subscription_id = "***"
tenant_id = "***"
client_id = "***"
client_secret = "***"

resource_group_name = '***'
credential = ClientSecretCredential(
    tenant_id=tenant_id,
    client_id=client_id,
    client_secret=client_secret
)
client = MySQLManagementClient(credential, subscription_id=subscription_id, logging_enable=True)


servers = client.servers.list_by_resource_group(resource_group_name)
server = next(servers)

Check the value of server and find publicNetworkAccess attribute.
Compare the results with the output from

az mysql flexible-server list --resource-group *** --debug --subscription ***

Compare the results with the output from HTTP API calls.

Obtain token

curl -X POST https://login.microsoftonline.com/<tenant_id>/oauth2/v2.0/token \
     -H "Content-Type: application/x-www-form-urlencoded" \
     -d "grant_type=client_credentials&client_id=<client_id>&client_secret=<client_secret>&scope=https%3A%2F%2Fmanagement.azure.com%2F.default"

Request list MySQL servers with api-version=2021-05-01
curl -X GET https://management.azure.com/subscriptions/<subscription_id>/resourceGroups/<resource_group_name>/providers/Microsoft.DBforMySQL/flexibleServers?api-version=2021-05-01 \
     -H "Authorization: Bearer <access_token>"
     
Now do the same for api-version=2024-02-01-preview
https://management.azure.com/subscriptions/<subscription_id>/resourceGroups/<resource_group_name>/providers/Microsoft.DBforMySQL/flexibleServers?api-version=2024-02-01-preview \
     -H "Authorization: Bearer <access_token>"

Expected behavior
I would expect to have the same data no matter which API and client is used. What happens is that Python SDK returns publicNetworkAccess=Enabled whereas latest az client and HTTP API calls to the latest version return publicNetworkAccess=Disabled.

Attaching response examples.
az_issue.txt

The text was updated successfully, but these errors were encountered:

xiangyan99 · 2024-07-12T14:42:55Z

Thanks for the feedback, we’ll investigate asap.

msyyc · 2024-07-15T00:24:45Z

@ChenxiJiang333 Please help on this issue

ChenxiJiang333 · 2024-07-15T02:06:33Z

got it

ChenxiJiang333 · 2024-07-17T08:21:34Z

Hi @antonkazlouski-orca
First, could you visit the Azure Portal to check and share the actual public access setting of your resource? For we need to confirm which behavior is incorrect.

The issue you mentioned that Python SDK makes different behavior from other interactive method seems impossible since Python SDK is also a tool to send/receive API calls, so it should return values just the same as HTTP API calls returns.
And by the log you provided, it seems more likely that you got the different outputs for you have called two different api versions: 2024-02-01-preview & 2021-05-01. Could you try get the server info again through Python SDK with specified api version 2024-02-01-preview by updating the code to:

client = MySQLManagementClient(credential, subscription_id=subscription_id, logging_enable=True, api_version="2024-02-01-preview")

and see whether there would be any change on the publicNetworkAccess's value?

github-actions · 2024-07-17T15:12:39Z

Hi @antonkazlouski-orca. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.

antonkazlouski-orca · 2024-07-22T07:23:47Z

Hi @xiangyan99 !

First, could you visit the Azure Portal to check and share the actual public access setting of your resource? For we need to confirm which behavior is incorrect.

I've requested this info as I'm unable to access the UI of the customer.

As for the rest, I've already tried to specify api_version in Python SDK but I don't see any effect of it.

Request URL: 'https://login.microsoftonline.com/***/v2.0/.well-known/openid-configuration'
Request method: 'GET'
Request headers:
    'User-Agent': 'azsdk-python-identity/1.12.0 Python/3.11.4 (Linux-5.10.219-208.866.amzn2.x86_64-x86_64-with-glibc2.36)'
No body was attached to the request
Request URL: 'https://login.microsoftonline.com/***/v2.0/.well-known/openid-configuration'
Request method: 'GET'
Request headers:
    'User-Agent': 'azsdk-python-identity/1.12.0 Python/3.11.4 (Linux-5.10.219-208.866.amzn2.x86_64-x86_64-with-glibc2.36)'
No body was attached to the request
Response status: 200
Response headers:
    'Cache-Control': 'max-age=86400, private'
    'Content-Type': 'application/json; charset=utf-8'
    'Strict-Transport-Security': 'REDACTED'
    'X-Content-Type-Options': 'REDACTED'
    'Access-Control-Allow-Origin': 'REDACTED'
    'Access-Control-Allow-Methods': 'REDACTED'
    'P3P': 'REDACTED'
    'x-ms-request-id': 'f47a437f-d023-4839-a8c5-58441ebe7f00'
    'x-ms-ests-server': 'REDACTED'
    'x-ms-srs': 'REDACTED'
    'X-XSS-Protection': 'REDACTED'
    'Set-Cookie': 'REDACTED'
    'Date': 'Mon, 22 Jul 2024 06:50:37 GMT'
    'Content-Length': '1753'
Response status: 200
Response headers:
    'Cache-Control': 'max-age=86400, private'
    'Content-Type': 'application/json; charset=utf-8'
    'Strict-Transport-Security': 'REDACTED'
    'X-Content-Type-Options': 'REDACTED'
    'Access-Control-Allow-Origin': 'REDACTED'
    'Access-Control-Allow-Methods': 'REDACTED'
    'P3P': 'REDACTED'
    'x-ms-request-id': 'f47a437f-d023-4839-a8c5-58441ebe7f00'
    'x-ms-ests-server': 'REDACTED'
    'x-ms-srs': 'REDACTED'
    'X-XSS-Protection': 'REDACTED'
    'Set-Cookie': 'REDACTED'
    'Date': 'Mon, 22 Jul 2024 06:50:37 GMT'
    'Content-Length': '1753'
Region configured (None) != region detected ('eu-central-1')
Request URL: 'https://login.microsoftonline.com/***/oauth2/v2.0/token'
Request method: 'POST'
Request headers:
    'Accept': 'application/json'
    'x-client-sku': 'REDACTED'
    'x-client-ver': 'REDACTED'
    'x-client-os': 'REDACTED'
    'x-client-cpu': 'REDACTED'
    'x-ms-lib-capability': 'REDACTED'
    'client-request-id': 'REDACTED'
    'x-client-current-telemetry': 'REDACTED'
    'x-client-last-telemetry': 'REDACTED'
    'User-Agent': 'azsdk-python-identity/1.12.0 Python/3.11.4 (Linux-5.10.219-208.866.amzn2.x86_64-x86_64-with-glibc2.36)'
A body is sent with the request
Request URL: 'https://login.microsoftonline.com/***/oauth2/v2.0/token'
Request method: 'POST'
Request headers:
    'Accept': 'application/json'
    'x-client-sku': 'REDACTED'
    'x-client-ver': 'REDACTED'
    'x-client-os': 'REDACTED'
    'x-client-cpu': 'REDACTED'
    'x-ms-lib-capability': 'REDACTED'
    'client-request-id': 'REDACTED'
    'x-client-current-telemetry': 'REDACTED'
    'x-client-last-telemetry': 'REDACTED'
    'User-Agent': 'azsdk-python-identity/1.12.0 Python/3.11.4 (Linux-5.10.219-208.866.amzn2.x86_64-x86_64-with-glibc2.36)'
A body is sent with the request
Response status: 200
Response headers:
    'Cache-Control': 'no-store, no-cache'
    'Pragma': 'no-cache'
    'Content-Type': 'application/json; charset=utf-8'
    'Expires': '-1'
    'Strict-Transport-Security': 'REDACTED'
    'X-Content-Type-Options': 'REDACTED'
    'P3P': 'REDACTED'
    'client-request-id': 'REDACTED'
    'x-ms-request-id': 'f60e9ede-c278-4251-b591-c20661f64b00'
    'x-ms-ests-server': 'REDACTED'
    'x-ms-clitelem': 'REDACTED'
    'x-ms-srs': 'REDACTED'
    'X-XSS-Protection': 'REDACTED'
    'Set-Cookie': 'REDACTED'
    'Date': 'Mon, 22 Jul 2024 06:50:38 GMT'
    'Content-Length': '1488'
Response status: 200
Response headers:
    'Cache-Control': 'no-store, no-cache'
    'Pragma': 'no-cache'
    'Content-Type': 'application/json; charset=utf-8'
    'Expires': '-1'
    'Strict-Transport-Security': 'REDACTED'
    'X-Content-Type-Options': 'REDACTED'
    'P3P': 'REDACTED'
    'client-request-id': 'REDACTED'
    'x-ms-request-id': 'f60e9ede-c278-4251-b591-c20661f64b00'
    'x-ms-ests-server': 'REDACTED'
    'x-ms-clitelem': 'REDACTED'
    'x-ms-srs': 'REDACTED'
    'X-XSS-Protection': 'REDACTED'
    'Set-Cookie': 'REDACTED'
    'Date': 'Mon, 22 Jul 2024 06:50:38 GMT'
    'Content-Length': '1488'
ClientSecretCredential.get_token succeeded
ClientSecretCredential.get_token succeeded
Request URL: 'https://management.azure.com/subscriptions/***/resourceGroups/***/providers/Microsoft.DBForMySql/flexibleServers?api-version=2020-07-01-preview'
Request method: 'GET'
Request headers:
    'Accept': 'application/json'
    'x-ms-client-request-id': 'b4419c20-47f6-11ef-9f08-f6888e3f0086'
    'User-Agent': 'azsdk-python-mgmt-rdbms/unknown Python/3.11.4 (Linux-5.10.219-208.866.amzn2.x86_64-x86_64-with-glibc2.36)'
    'Authorization': 'Bearer ***'
Request body:
This request has no body
Request URL: 'https://management.azure.com/subscriptions/***/resourceGroups/***/providers/Microsoft.DBForMySql/flexibleServers?api-version=2020-07-01-preview'
Request method: 'GET'
Request headers:
    'Accept': 'application/json'
    'x-ms-client-request-id': 'b4419c20-47f6-11ef-9f08-f6888e3f0086'
    'User-Agent': 'azsdk-python-mgmt-rdbms/unknown Python/3.11.4 (Linux-5.10.219-208.866.amzn2.x86_64-x86_64-with-glibc2.36)'
    'Authorization': 'Bearer ***'
Request body:
This request has no body
Request URL: 'https://management.azure.com/subscriptions/***/resourceGroups/***/providers/Microsoft.DBForMySql/flexibleServers?api-version=REDACTED'
Request method: 'GET'
Request headers:
    'Accept': 'application/json'
    'x-ms-client-request-id': 'b4419c20-47f6-11ef-9f08-f6888e3f0086'
    'User-Agent': 'azsdk-python-mgmt-rdbms/unknown Python/3.11.4 (Linux-5.10.219-208.866.amzn2.x86_64-x86_64-with-glibc2.36)'
    'Authorization': 'REDACTED'
No body was attached to the request
Request URL: 'https://management.azure.com/subscriptions/***/resourceGroups/***/providers/Microsoft.DBForMySql/flexibleServers?api-version=REDACTED'
Request method: 'GET'
Request headers:
    'Accept': 'application/json'
    'x-ms-client-request-id': 'b4419c20-47f6-11ef-9f08-f6888e3f0086'
    'User-Agent': 'azsdk-python-mgmt-rdbms/unknown Python/3.11.4 (Linux-5.10.219-208.866.amzn2.x86_64-x86_64-with-glibc2.36)'
    'Authorization': 'REDACTED'
No body was attached to the request
Response status: 200
Response headers:
    'Cache-Control': 'no-cache'
    'Pragma': 'no-cache'
    'Content-Length': '1331'
    'Content-Type': 'application/json; charset=utf-8'
    'Expires': '-1'
    'X-Content-Type-Options': 'REDACTED'
    'x-ms-request-id': '670a55d1-d434-45fb-b9ef-13a4bb4d5110'
    'x-ms-ratelimit-remaining-subscription-reads': 'REDACTED'
    'x-ms-ratelimit-remaining-subscription-global-reads': 'REDACTED'
    'x-ms-correlation-request-id': 'REDACTED'
    'x-ms-routing-request-id': 'REDACTED'
    'Strict-Transport-Security': 'REDACTED'
    'X-Cache': 'REDACTED'
    'X-MSEdge-Ref': 'REDACTED'
    'Date': 'Mon, 22 Jul 2024 06:50:38 GMT'
Response status: 200
Response headers:
    'Cache-Control': 'no-cache'
    'Pragma': 'no-cache'
    'Content-Length': '1331'
    'Content-Type': 'application/json; charset=utf-8'
    'Expires': '-1'
    'X-Content-Type-Options': 'REDACTED'
    'x-ms-request-id': '670a55d1-d434-45fb-b9ef-13a4bb4d5110'
    'x-ms-ratelimit-remaining-subscription-reads': 'REDACTED'
    'x-ms-ratelimit-remaining-subscription-global-reads': 'REDACTED'
    'x-ms-correlation-request-id': 'REDACTED'
    'x-ms-routing-request-id': 'REDACTED'
    'Strict-Transport-Security': 'REDACTED'
    'X-Cache': 'REDACTED'
    'X-MSEdge-Ref': 'REDACTED'
    'Date': 'Mon, 22 Jul 2024 06:50:38 GMT'
Response status: '200'
Response headers:
    'Cache-Control': 'no-cache'
    'Pragma': 'no-cache'
    'Content-Length': '1331'
    'Content-Type': 'application/json; charset=utf-8'
    'Expires': '-1'
    'X-Content-Type-Options': 'nosniff'
    'x-ms-request-id': '670a55d1-d434-45fb-b9ef-13a4bb4d5110'
    'x-ms-ratelimit-remaining-subscription-reads': '249'
    'x-ms-ratelimit-remaining-subscription-global-reads': '3749'
    'x-ms-correlation-request-id': 'd58f5b95-5b85-4dc1-859c-8debf9fe05dd'
    'x-ms-routing-request-id': 'GERMANYWESTCENTRAL:20240722T065039Z:d58f5b95-5b85-4dc1-859c-8debf9fe05dd'
    'Strict-Transport-Security': 'max-age=31536000; includeSubDomains'
    'X-Cache': 'CONFIG_NOCACHE'
    'X-MSEdge-Ref': 'Ref A: F93BB325522B48F3B8639626C4EA6435 Ref B: VIEEDGE2409 Ref C: 2024-07-22T06:50:39Z'
    'Date': 'Mon, 22 Jul 2024 06:50:38 GMT'
Response content:
{"value":[{"sku":{"name":"Standard_D2ds_v4","tier":"GeneralPurpose"},"properties":{"administratorLogin":"***","storageProfile":{"storageMB":131072,"storageIops":684,"backupRetentionDays":7,"storageAutogrow":"Enabled","fileStorageSkuName":"Premium_LRS"},"version":"8.0.21","state":"Ready","haState":"NotEnabled","fullyQualifiedDomainName":"***","sourceServerId":"","publicNetworkAccess":"Enabled","sslEnforcement":"Disabled","haEnabled":"Disabled","availabilityZone":"3","standbyAvailabilityZone":"","earliestRestoreDate":"2024-07-15T06:50:39.4785006+00:00","maintenanceWindow":{"customWindow":"Disabled","dayOfWeek":0,"startHour":0,"startMinute":0},"replicationRole":"None","replicaCapacity":10,"byokEnforcement":"Disabled"},"location":"West Europe","tags":{"BusinessOwnerEmail":"***","CostCenter":"***","Criticality":"Low","Department":"***","DevOwnerEmail":"***","MaintenanceWindow":"3rd Sun 10-12am CET","ProjectName":"***","ApplicationUID":"***"},"id":"/subscriptions/***/resourceGroups/***/providers/Microsoft.DBforMySQL/flexibleServers/***","name":"***","type":"Microsoft.DBforMySQL/flexibleServers"}]}
Response status: '200'
Response headers:
    'Cache-Control': 'no-cache'
    'Pragma': 'no-cache'
    'Content-Length': '1331'
    'Content-Type': 'application/json; charset=utf-8'
    'Expires': '-1'
    'X-Content-Type-Options': 'nosniff'
    'x-ms-request-id': '670a55d1-d434-45fb-b9ef-13a4bb4d5110'
    'x-ms-ratelimit-remaining-subscription-reads': '249'
    'x-ms-ratelimit-remaining-subscription-global-reads': '3749'
    'x-ms-correlation-request-id': 'd58f5b95-5b85-4dc1-859c-8debf9fe05dd'
    'x-ms-routing-request-id': 'GERMANYWESTCENTRAL:20240722T065039Z:d58f5b95-5b85-4dc1-859c-8debf9fe05dd'
    'Strict-Transport-Security': 'max-age=31536000; includeSubDomains'
    'X-Cache': 'CONFIG_NOCACHE'
    'X-MSEdge-Ref': 'Ref A: F93BB325522B48F3B8639626C4EA6435 Ref B: VIEEDGE2409 Ref C: 2024-07-22T06:50:39Z'
    'Date': 'Mon, 22 Jul 2024 06:50:38 GMT'
Response content:
{"value":[{"sku":{"name":"Standard_D2ds_v4","tier":"GeneralPurpose"},"properties":{"administratorLogin":"***","storageProfile":{"storageMB":131072,"storageIops":684,"backupRetentionDays":7,"storageAutogrow":"Enabled","fileStorageSkuName":"Premium_LRS"},"version":"8.0.21","state":"Ready","haState":"NotEnabled","fullyQualifiedDomainName":"***","sourceServerId":"","publicNetworkAccess":"Enabled","sslEnforcement":"Disabled","haEnabled":"Disabled","availabilityZone":"3","standbyAvailabilityZone":"","earliestRestoreDate":"2024-07-15T06:50:39.4785006+00:00","maintenanceWindow":{"customWindow":"Disabled","dayOfWeek":0,"startHour":0,"startMinute":0},"replicationRole":"None","replicaCapacity":10,"byokEnforcement":"Disabled"},"location":"West Europe","tags":{"BusinessOwnerEmail":"***","CostCenter":"***","Criticality":"Low","Department":"***","DevOwnerEmail":"***","MaintenanceWindow":"3rd Sun 10-12am CET","ProjectName":"***","ApplicationUID":"***"},"id":"/subscriptions/***/resourceGroups/***/providers/Microsoft.DBforMySQL/flexibleServers/***","name":"***","type":"Microsoft.DBforMySQL/flexibleServers"}]}

In [4]:

In the Request URL attribute there is still api-version=2020-07-01-preview. It appears that the api_version is hardcoded in the latest stable release. However, it is not on the master branch.

ChenxiJiang333 · 2024-07-22T09:32:35Z

Hi @antonkazlouski-orca, please try to use azure-mgmt-rdbms==10.2.0b17 to run the codes instead.

github-actions · 2024-07-22T14:54:55Z

Hi @antonkazlouski-orca. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.

antonkazlouski-orca · 2024-07-22T15:54:47Z

@ChenxiJiang333 that worked, but we won't consider using beta package in the production system. Is there any workaround?

msyyc · 2024-07-23T02:33:08Z

@ChenxiJiang333 that worked, but we won't consider using beta package in the production system. Is there any workaround?

azure-mgmt-rdbms is a complex package which contains several service so it keeps beta as long as not all service is stable. SDK PM @josefree is working on split of the package.

antonkazlouski-orca · 2024-07-23T06:58:33Z

@msyyc thank you. So, it leaves us with 2 options:

Use beta version
Patch the existing version ourselves and use it until a stable release is ready

Correct?

ChenxiJiang333 · 2024-07-23T10:05:22Z

@msyyc thank you. So, it leaves us with 2 options:

Use beta version

Patch the existing version ourselves and use it until a stable release is ready

Correct?

Yes, for now, we cannot make a new stable version immediately.

xiangyan99 added Mgmt This issue is related to a management-plane library. RDBMS and removed needs-triage This is a new issue that needs to be triaged to the appropriate team. labels Jul 12, 2024

xiangyan99 assigned msyyc Jul 12, 2024

github-actions bot added the needs-team-attention This issue needs attention from Azure service team or SDK team label Jul 12, 2024

xiangyan99 added the needs-author-feedback More information is needed from author to address the issue. label Jul 17, 2024

github-actions bot removed the needs-team-attention This issue needs attention from Azure service team or SDK team label Jul 17, 2024

github-actions bot added needs-team-attention This issue needs attention from Azure service team or SDK team and removed needs-author-feedback More information is needed from author to address the issue. labels Jul 22, 2024

xiangyan99 added the needs-author-feedback More information is needed from author to address the issue. label Jul 22, 2024

github-actions bot removed the needs-team-attention This issue needs attention from Azure service team or SDK team label Jul 22, 2024

github-actions bot added needs-team-attention This issue needs attention from Azure service team or SDK team and removed needs-author-feedback More information is needed from author to address the issue. labels Jul 22, 2024

antonkazlouski-orca closed this as completed Jul 23, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

The latest azure.mgmt.rdbms returns publicNetworkAccess = Enabled whereas latest az cli returns publicNetworkAccess = Disabled #36459

The latest azure.mgmt.rdbms returns publicNetworkAccess = Enabled whereas latest az cli returns publicNetworkAccess = Disabled #36459

antonkazlouski-orca commented Jul 12, 2024

xiangyan99 commented Jul 12, 2024

msyyc commented Jul 15, 2024

ChenxiJiang333 commented Jul 15, 2024

ChenxiJiang333 commented Jul 17, 2024 •

edited

Loading

github-actions bot commented Jul 17, 2024

antonkazlouski-orca commented Jul 22, 2024

ChenxiJiang333 commented Jul 22, 2024 •

edited

Loading

github-actions bot commented Jul 22, 2024

antonkazlouski-orca commented Jul 22, 2024

msyyc commented Jul 23, 2024

antonkazlouski-orca commented Jul 23, 2024

ChenxiJiang333 commented Jul 23, 2024

The latest azure.mgmt.rdbms returns publicNetworkAccess = Enabled whereas latest az cli returns publicNetworkAccess = Disabled #36459

The latest azure.mgmt.rdbms returns publicNetworkAccess = Enabled whereas latest az cli returns publicNetworkAccess = Disabled #36459

Comments

antonkazlouski-orca commented Jul 12, 2024

xiangyan99 commented Jul 12, 2024

msyyc commented Jul 15, 2024

ChenxiJiang333 commented Jul 15, 2024

ChenxiJiang333 commented Jul 17, 2024 • edited Loading

github-actions bot commented Jul 17, 2024

antonkazlouski-orca commented Jul 22, 2024

ChenxiJiang333 commented Jul 22, 2024 • edited Loading

github-actions bot commented Jul 22, 2024

antonkazlouski-orca commented Jul 22, 2024

msyyc commented Jul 23, 2024

antonkazlouski-orca commented Jul 23, 2024

ChenxiJiang333 commented Jul 23, 2024

ChenxiJiang333 commented Jul 17, 2024 •

edited

Loading

ChenxiJiang333 commented Jul 22, 2024 •

edited

Loading