feat: add cli to test other (custom) endpoints (#46)

* feat: add cli to test other (custom) endpoints Should make it easier to develop and verify, e.g., public endpoints * fix: adapt to the us vs rest domain in cli test * fix(mypy): naming * fix: exclude docs build for mypy
encord-team · Jan 15, 2025 · b47adcb · b47adcb
1 parent 49364fd
commit b47adcb
Show file tree

Hide file tree

Showing 10 changed files with 111 additions and 71 deletions.
diff --git a/..._examples/fastapi/frame_classification.py → ...s/fastapi/fastapi_frame_classification.py b/..._examples/fastapi/frame_classification.py → ...s/fastapi/fastapi_frame_classification.py
diff --git a/...examples/fastapi/object_classification.py → .../fastapi/fastapi_object_classification.py b/...examples/fastapi/object_classification.py → .../fastapi/fastapi_object_classification.py
diff --git a/docs/code_examples/gcp/add_bounding_box.py → ...code_examples/gcp/gcp_add_bounding_box.py b/docs/code_examples/gcp/add_bounding_box.py → ...code_examples/gcp/gcp_add_bounding_box.py
diff --git a/...code_examples/gcp/frame_classification.py → ..._examples/gcp/gcp_frame_classification.py b/...code_examples/gcp/frame_classification.py → ..._examples/gcp/gcp_frame_classification.py
diff --git a/...ode_examples/gcp/object_classification.py → ...examples/gcp/gcp_object_classification.py b/...ode_examples/gcp/object_classification.py → ...examples/gcp/gcp_object_classification.py
diff --git a/docs/editor_agents/examples/index.md b/docs/editor_agents/examples/index.md
@@ -128,7 +128,7 @@ An agent that transforms a labeling task from Figure A to Figure B, as shown bel
 
 ??? "The full code for `agent.py`"
     <!--codeinclude-->
-    [agent.py](../../code_examples/gcp/frame_classification.py) linenums:1
+    [agent.py](../../code_examples/gcp/gcp_frame_classification.py) linenums:1
     <!--/codeinclude-->
 
 Let's go through the code section by section.
@@ -139,13 +139,13 @@ First, we import dependencies and set up the Project:
     Make sure to insert your Project's hash here.
 
 <!--codeinclude-->
-[agent.py](../../code_examples/gcp/frame_classification.py) lines:1-15
+[agent.py](../../code_examples/gcp/gcp_frame_classification.py) lines:1-15
 <!--/codeinclude-->
 
 Next, we create a data model and a system prompt based on the Project Ontology that will tell Claude how to structure its response:
 
 <!--codeinclude-->
-[agent.py](../../code_examples/gcp/frame_classification.py) lines:18-29
+[agent.py](../../code_examples/gcp/gcp_frame_classification.py) lines:18-29
 <!--/codeinclude-->
 
 
@@ -316,14 +316,14 @@ Next, we create a data model and a system prompt based on the Project Ontology t
 We also need an Anthropic API client to communicate with Claude:
 
 <!--codeinclude-->
-[agent.py](../../code_examples/gcp/frame_classification.py) lines:32-33
+[agent.py](../../code_examples/gcp/gcp_frame_classification.py) lines:32-33
 <!--/codeinclude-->
 
 
 Finally, we define our editor agent:
 
 <!--codeinclude-->
-[agent.py](../../code_examples/gcp/frame_classification.py) lines:36-65
+[agent.py](../../code_examples/gcp/gcp_frame_classification.py) lines:36-65
 <!--/codeinclude-->
 
 The agent:  
@@ -587,7 +587,7 @@ The goal is to be able to trigger an agent that takes a labeling task from Figur
 ??? "The full code for `agent.py`"
     <!--codeinclude-->
 
-    [agent.py](../../code_examples/gcp/object_classification.py) linenums:1
+    [agent.py](../../code_examples/gcp/gcp_object_classification.py) linenums:1
 
     <!--/codeinclude-->
 
@@ -597,15 +597,15 @@ For this, you will need to have your `<project_hash>` ready.
 
 <!--codeinclude-->
 
-[agent.py](../../code_examples/gcp/object_classification.py) lines:1-14
+[agent.py](../../code_examples/gcp/gcp_object_classification.py) lines:1-14
 
 <!--/codeinclude-->
 
 Now that we have the project, we can extract the generic ontology object as well as that actual ontology objects that we care about.
 
 <!--codeinclude-->
 
-[agent.py](../../code_examples/gcp/object_classification.py) lines:15-19
+[agent.py](../../code_examples/gcp/gcp_object_classification.py) lines:15-19
 
 <!--/codeinclude-->
 
@@ -621,7 +621,7 @@ is only allowed to choose between the object types that are not of the generic o
 
 <!--codeinclude-->
 
-[agent.py](../../code_examples/gcp/object_classification.py) lines:22-30
+[agent.py](../../code_examples/gcp/gcp_object_classification.py) lines:22-30
 
 <!--/codeinclude-->
 
@@ -905,15 +905,15 @@ With the system prompt ready, we can instantiate an api client for Claude.
 
 <!--codeinclude-->
 
-[agent.py](../../code_examples/gcp/object_classification.py) lines:33-34
+[agent.py](../../code_examples/gcp/gcp_object_classification.py) lines:33-34
 
 <!--/codeinclude-->
 
 Now, let's define the editor agent.
 
 <!--codeinclude-->
 
-[agent.py](../../code_examples/gcp/object_classification.py) lines:38-46
+[agent.py](../../code_examples/gcp/gcp_object_classification.py) lines:38-46
 
 <!--/codeinclude-->
 
@@ -927,7 +927,7 @@ Notice how the `crop` variable has a convenient `b64_encoding` method to produce
 
 <!--codeinclude-->
 
-[agent.py](../../code_examples/gcp/object_classification.py) lines:47-60
+[agent.py](../../code_examples/gcp/gcp_object_classification.py) lines:47-60
 
 <!--/codeinclude-->
 
@@ -938,15 +938,15 @@ If successful, the old generic object can be removed and the newly classified ob
 
 <!--codeinclude-->
 
-[agent.py](../../code_examples/gcp/object_classification.py) lines:63-80
+[agent.py](../../code_examples/gcp/gcp_object_classification.py) lines:63-80
 
 <!--/codeinclude-->
 
 Finally, we'll save the labels with Encord.
 
 <!--codeinclude-->
 
-[agent.py](../../code_examples/gcp/object_classification.py) lines:83-84
+[agent.py](../../code_examples/gcp/gcp_object_classification.py) lines:83-84
 
 <!--/codeinclude-->
 
@@ -1039,35 +1039,35 @@ The goal is to trigger an agent that takes a labeling task from Figure A to Figu
 
 ??? "The full code for `main.py`"
     <!--codeinclude-->
-    [main.py](../../code_examples/fastapi/frame_classification.py) linenums:1
+    [main.py](../../code_examples/fastapi/fastapi_frame_classification.py) linenums:1
     <!--/codeinclude-->
 
 Let us go through the code section by section.
 
 First, we import dependencies and setup the FastAPI app with CORS middleware:
 
 <!--codeinclude-->
-[main.py](../../code_examples/fastapi/frame_classification.py) lines:1-22
+[main.py](../../code_examples/fastapi/fastapi_frame_classification.py) lines:1-22
 <!--/codeinclude-->
 
 The CORS middleware is crucial as it allows the Encord platform to make requests to your API.
 
 Next, we set up the Project and create a data model based on the Ontology:
 
 <!--codeinclude-->
-[main.py](../../code_examples/fastapi/frame_classification.py) lines:25-27
+[main.py](../../code_examples/fastapi/fastapi_frame_classification.py) lines:25-27
 <!--/codeinclude-->
 
 We create the system prompt that tells Claude how to structure its response:
 
 <!--codeinclude-->
-[main.py](../../code_examples/fastapi/frame_classification.py) lines:30-42
+[main.py](../../code_examples/fastapi/fastapi_frame_classification.py) lines:30-42
 <!--/codeinclude-->
 
 Finally, we define the endpoint to handle the classification:
 
 <!--codeinclude-->
-[main.py](../../code_examples/fastapi/frame_classification.py) lines:45-75
+[main.py](../../code_examples/fastapi/fastapi_frame_classification.py) lines:45-75
 <!--/codeinclude-->
 
 The endpoint:
@@ -1147,33 +1147,33 @@ The goal is to trigger an agent that takes a labeling task from Figure A to Figu
 
 ??? "The full code for `main.py`"
     <!--codeinclude-->
-    [main.py](../../code_examples/fastapi/object_classification.py) linenums:1
+    [main.py](../../code_examples/fastapi/fastapi_object_classification.py) linenums:1
     <!--/codeinclude-->
 
 Let's walk through the key components.
 
 First, we setup the FastAPI app and CORS middleware:
 
 <!--codeinclude-->
-[main.py](../../code_examples/fastapi/object_classification.py) lines:1-20
+[main.py](../../code_examples/fastapi/fastapi_object_classification.py) lines:1-20
 <!--/codeinclude-->
 
 Then we setup the client, Project, and extract the generic Ontology object:
 
 <!--codeinclude-->
-[main.py](../../code_examples/fastapi/object_classification.py) lines:23-29
+[main.py](../../code_examples/fastapi/fastapi_object_classification.py) lines:23-29
 <!--/codeinclude-->
 
 We create the data model and system prompt for Claude:
 
 <!--codeinclude-->
-[main.py](../../code_examples/fastapi/object_classification.py) lines:32-44
+[main.py](../../code_examples/fastapi/fastapi_object_classification.py) lines:32-44
 <!--/codeinclude-->
 
 Finally, we define our object classification endpoint:
 
 <!--codeinclude-->
-[main.py](../../code_examples/fastapi/object_classification.py) lines:47-94
+[main.py](../../code_examples/fastapi/fastapi_object_classification.py) lines:47-94
 <!--/codeinclude-->
 
 The endpoint:

diff --git a/encord_agents/cli/test.py b/encord_agents/cli/test.py
@@ -3,12 +3,20 @@
 """
 
 import os
+import re
+import sys
 
+import requests
+import rich
+import typer
 from rich.progress import Progress, SpinnerColumn, TextColumn, TimeElapsedColumn
 from rich.table import Table
 from typer import Argument, Option, Typer
 from typing_extensions import Annotated
 
+from encord_agents import FrameData
+from encord_agents.core.constants import EDITOR_URL_PARTS_REGEX
+
 app = Typer(
     name="test",
     help="Utility for testing agents",
@@ -17,67 +25,40 @@
 )
 
 
-@app.command(
-    "local",
-    short_help="Hit a localhost agents endpoint for testing",
-)
-def local(
-    target: Annotated[
-        str,
-        Argument(help="Name of the localhost endpoint to hit ('http://localhost/{target}')"),
-    ],
-    url: Annotated[str, Argument(help="Url copy/pasted from label editor")],
-    port: Annotated[int, Option(help="Local host port to hit")] = 8080,
-) -> None:
-    """Hit a localhost agents endpoint for testing an agent by copying the url from the Encord Label Editor over.
-
-    Given
-
-        - A url of the form [blue]`https://app.encord.com/label_editor/[green]{project_hash}[/green]/[green]{data_hash}[/green]/[green]{frame}[/green]`[/blue]
-        - A [green]target[/green] endpoint
-        - A [green]port[/green] (optional)
-
-    The url [blue]http://localhost:[green]{port}[/green]/[green]{target}[/green][/blue] will be hit with a post request containing:
-    {
-        "projectHash": "[green]{project_hash}[/green]",
-        "dataHash": "[green]{data_hash}[/green]",
-        "frame": [green]frame[/green] or 0
-    }
+def parse_editor_url(editor_url: str) -> tuple[FrameData, str]:
     """
-    import re
-    import sys
-    from pprint import pprint
+    Reads project_hash, data_hash, frame and domain from the editor url.
 
-    import requests
-    import rich
-    import typer
-
-    parts_regex = r"https:\/\/app.encord.com\/label_editor\/(?P<projectHash>.*?)\/(?P<dataHash>[\w\d]{8}-[\w\d]{4}-[\w\d]{4}-[\w\d]{4}-[\w\d]{12})(/(?P<frame>\d+))?\??"
+    Args:
+        - editor_url: The url obtained from the Label Editor.
 
+    Returns:
+        The FrameData object and the domain of the url.
+    """
     try:
-        match = re.match(parts_regex, url)
+        match = re.match(EDITOR_URL_PARTS_REGEX, editor_url)
         if match is None:
             raise typer.Abort()
-
         payload = match.groupdict()
+        domain = payload.pop("domain")
         payload["frame"] = payload["frame"] or 0
+        return FrameData.model_validate(payload), domain
     except Exception:
         rich.print(
             """Could not match url to the expected format.
-Format is expected to be [blue]https://app.encord.com/label_editor/[magenta]{project_hash}[/magenta]/[magenta]{data_hash}[/magenta](/[magenta]{frame}[/magenta])[/blue]
+Format is expected to be [blue]https://app.(us.)?encord.com/label_editor/[magenta]{project_hash}[/magenta]/[magenta]{data_hash}[/magenta](/[magenta]{frame}[/magenta])[/blue]
 """,
             file=sys.stderr,
         )
         raise typer.Abort()
 
-    if target and not target[0] == "/":
-        target = f"/{target}"
 
+def hit_endpoint(endpoint: str, payload: FrameData, domain: str) -> None:
     with requests.Session() as sess:
         request = requests.Request(
             "POST",
-            f"http://localhost:{port}{target}",
-            json=payload,
+            endpoint,
+            json=payload.model_dump(mode="json", by_alias=True),
             headers={"Content-type": "application/json"},
         )
         prepped = request.prepare()
@@ -109,9 +90,7 @@ def local(
 
         table.add_section()
         table.add_row("[green]Utilities[/green]")
-        editor_url = (
-            f"https://app.encord.com/label_editor/{payload['projectHash']}/{payload['dataHash']}/{payload['frame']}"
-        )
+        editor_url = f"{domain}/label_editor/{payload.project_hash}/{payload.data_hash}/{payload.frame}"
         table.add_row("label editor", editor_url)
 
         headers = ["'{0}: {1}'".format(k, v) for k, v in prepped.headers.items()]
@@ -120,3 +99,63 @@ def local(
         table.add_row("curl", curl_command)
 
         rich.print(table)
+
+
+@app.command("custom", short_help="Hit a custom endpoint for testing purposes")
+def custom(
+    endpoint: Annotated[str, Argument(help="Endpoint to hit with json payload")],
+    editor_url: Annotated[str, Argument(help="Url copy/pasted from label editor")],
+) -> None:
+    """
+    Hit a custom agents endpoint for testing an editor agent by copying the url from the Encord Label Editor.
+
+    Given
+
+        - The endpoint you wish to test
+        - An editor url of the form [blue]`https://app.encord.com/label_editor/[green]{project_hash}[/green]/[green]{data_hash}[/green]/[green]{frame}[/green]`[/blue]
+        - A [green]port[/green] (optional)
+
+    The url [blue]http://localhost:[green]{port}[/green]/[green]{target}[/green][/blue] will be hit with a post request containing:
+    {
+        "projectHash": "[green]{project_hash}[/green]",
+        "dataHash": "[green]{data_hash}[/green]",
+        "frame": [green]frame[/green] or 0
+    }
+    """
+    payload, domain = parse_editor_url(editor_url)
+    hit_endpoint(endpoint, payload, domain)
+
+
+@app.command(
+    "local",
+    short_help="Hit a localhost agents endpoint for testing",
+)
+def local(
+    target: Annotated[
+        str,
+        Argument(help="Name of the localhost endpoint to hit ('http://localhost/{target}')"),
+    ],
+    editor_url: Annotated[str, Argument(help="Url copy/pasted from label editor")],
+    port: Annotated[int, Option(help="Local host port to hit")] = 8080,
+) -> None:
+    """Hit a localhost agents endpoint for testing an agent by copying the url from the Encord Label Editor over.
+
+    Given
+
+        - An editor url of the form [blue]`https://app.encord.com/label_editor/[green]{project_hash}[/green]/[green]{data_hash}[/green]/[green]{frame}[/green]`[/blue]
+        - A [green]port[/green] (optional)
+
+    The url [blue]http://localhost:[green]{port}[/green]/[green]{target}[/green][/blue] will be hit with a post request containing:
+    {
+        "projectHash": "[green]{project_hash}[/green]",
+        "dataHash": "[green]{data_hash}[/green]",
+        "frame": [green]frame[/green] or 0
+    }
+    """
+    payload, domain = parse_editor_url(editor_url)
+
+    if target and not target[0] == "/":
+        target = f"/{target}"
+    endpoint = f"http://localhost:{port}{target}"
+
+    hit_endpoint(endpoint, payload, domain)
diff --git a/encord_agents/core/constants.py b/encord_agents/core/constants.py
@@ -1,3 +1,4 @@
 ENCORD_DOMAIN_REGEX = (
     r"^https:\/\/(?:(?:cord-ai-development--[\w\d]+-[\w\d]+\.web.app)|(?:(?:dev|staging|app)\.(us\.)?encord\.com))$"
 )
+EDITOR_URL_PARTS_REGEX = r"(?<domain>https:\/\/app.(us\.)?encord.com)\/label_editor\/(?P<projectHash>.*?)\/(?P<dataHash>[\w\d]{8}-[\w\d]{4}-[\w\d]{4}-[\w\d]{4}-[\w\d]{12})(\/(?P<frame>\d+))?\??"
diff --git a/encord_agents/core/data_model.py b/encord_agents/core/data_model.py
@@ -45,11 +45,11 @@ class FrameData(BaseModel):
     Holds the data sent from the Encord Label Editor at the time of triggering the agent.
     """
 
-    project_hash: UUID = Field(validation_alias="projectHash")
+    project_hash: UUID = Field(alias="projectHash")
     """
     The identifier of the given project.
     """
-    data_hash: UUID = Field(validation_alias="dataHash")
+    data_hash: UUID = Field(alias="dataHash")
     """
     The identifier of the given data asset.
     """