Skip to content

[General Inference] Improve lifecycle messages #84

@johncalesp

Description

@johncalesp

Right new, I can create a General Inference endpoint using get_centml_client().
If I use create_inference it gives the correct response:

id=3889 created_at=datetime.datetime(2025, 1, 22, 15, 34, 40, 417631, tzinfo=TzInfo(UTC)) endpoint_url='john-test.e0ccbae3.gcp.centml.org'

Now, we need to add more friendly messages for getting the status, and when deleting.
For example, if I delete the endpoint using delete(), I get None as response:

>>> with get_centml_client() as cclient:
...     print(cclient.delete(3891))
... 
None>>> with get_centml_client() as cclient:
...     print(cclient.delete(3891))
... 
None

Here we should confirm what we deleted the endpoint, give the id and saying that the request was sucessfull

Another point to consider is retrieving the status of the endpoint after deleting. As of now it raises an error:
for get_inference()

>>> with get_centml_client() as cclient:
...     print(cclient.get_inference(3891))
... 
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "/home/john/Documents/centml/platform/centml-python-client/centml/sdk/api.py", line 28, in get_inference
    return self._api.get_inference_deployment_deployments_inference_deployment_id_get(id)
  File "/home/john/Documents/centml/platform/centml-python-client/venv/lib/python3.10/site-packages/pydantic/_internal/_validate_call.py", line 38, in wrapper_function
    return wrapper(*args, **kwargs)
  File "/home/john/Documents/centml/platform/centml-python-client/venv/lib/python3.10/site-packages/pydantic/_internal/_validate_call.py", line 111, in __call__
    res = self.__pydantic_validator__.validate_python(pydantic_core.ArgsKwargs(args, kwargs))
  File "/home/john/Documents/centml/platform/centml-python-client/venv/lib/python3.10/site-packages/platform_api_python_client/api/external_api.py", line 4717, in get_inference_deployment_deployments_inference_deployment_id_get
    return self.api_client.response_deserialize(
  File "/home/john/Documents/centml/platform/centml-python-client/venv/lib/python3.10/site-packages/platform_api_python_client/api_client.py", line 322, in response_deserialize
    raise ApiException.from_response(
  File "/home/john/Documents/centml/platform/centml-python-client/venv/lib/python3.10/site-packages/platform_api_python_client/exceptions.py", line 151, in from_response
    raise NotFoundException(http_resp=http_resp, body=body, data=data)
platform_api_python_client.exceptions.NotFoundException: (404)
Reason: Not Found
HTTP response headers: HTTPHeaderDict({'date': 'Wed, 22 Jan 2025 15:57:36 GMT', 'server': 'istio-envoy', 'content-length': '38', 'content-type': 'application/json', 'x-envoy-upstream-service-time': '11'})
HTTP response body: {"detail":"Deployment does not exist"}

For get_status()

>>> with get_centml_client() as cclient:
...     print(cclient.get_status(3891))
... 
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "/home/john/Documents/centml/platform/centml-python-client/centml/sdk/api.py", line 25, in get_status
    return self._api.get_deployment_status_deployments_status_deployment_id_get(id)
  File "/home/john/Documents/centml/platform/centml-python-client/venv/lib/python3.10/site-packages/pydantic/_internal/_validate_call.py", line 38, in wrapper_function
    return wrapper(*args, **kwargs)
  File "/home/john/Documents/centml/platform/centml-python-client/venv/lib/python3.10/site-packages/pydantic/_internal/_validate_call.py", line 111, in __call__
    res = self.__pydantic_validator__.validate_python(pydantic_core.ArgsKwargs(args, kwargs))
  File "/home/john/Documents/centml/platform/centml-python-client/venv/lib/python3.10/site-packages/platform_api_python_client/api/external_api.py", line 3879, in get_deployment_status_deployments_status_deployment_id_get
    return self.api_client.response_deserialize(
  File "/home/john/Documents/centml/platform/centml-python-client/venv/lib/python3.10/site-packages/platform_api_python_client/api_client.py", line 322, in response_deserialize
    raise ApiException.from_response(
  File "/home/john/Documents/centml/platform/centml-python-client/venv/lib/python3.10/site-packages/platform_api_python_client/exceptions.py", line 151, in from_response
    raise NotFoundException(http_resp=http_resp, body=body, data=data)
platform_api_python_client.exceptions.NotFoundException: (404)
Reason: Not Found
HTTP response headers: HTTPHeaderDict({'date': 'Wed, 22 Jan 2025 15:58:05 GMT', 'server': 'istio-envoy', 'content-length': '38', 'content-type': 'application/json', 'x-envoy-upstream-service-time': '18'})
HTTP response body: {"detail":"Deployment does not exist"}

Based on this information, I suggest the following:

  1. Return correct message when deleting endpoint
  2. Give correct status messages after endpoint is deleted and not raise an error

┆Issue is synchronized with this Notion page by Unito

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions