Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add hugie config modify sub-command (resolves #41) #47

Merged
merged 9 commits into from
Jan 11, 2023
File renamed without changes.
5 changes: 3 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
# Venv, direnv
.env
.envrc
.venv/

# Python cache, eggs
*egg-info/
*__pycache__/
dist/
*egg-info/
build
dist/
3 changes: 2 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]

### Added
- Add endpoint config subcommand #47 (resolves #41)
- Adds `ls` alias to `list` command #50
- Adds `-f` shortcut to `--force` command #50

Expand All @@ -32,5 +33,5 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Make delete error messsage more informative #25
- Add tests and CI/CD

### Bug fixes
### Fixed
- Correct table column widths #16
59 changes: 59 additions & 0 deletions docs/config.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# Config

This page details the usage of the config command

```
Usage: hugie config [OPTIONS] COMMAND [ARGS]...

╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --help Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Commands ───────────────────────────────────────────────────────────────────╮
│ modify Modify an existing endpoint config file │
╰──────────────────────────────────────────────────────────────────────────────╯
```

## Modify

Modify an existing config file.

Command referene:
```
Usage: hugie config modify [OPTIONS] PATH

Modify an existing endpoint config file

╭─ Arguments ──────────────────────────────────────────────────────────────────╮
│ * path TEXT [default: None] [required] │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --accountid TEXT ID of the account (for private endpoints) │
│ [default: None] │
│ --name TEXT Name of the endpoint [default: None] │
│ --type TEXT Type of endpoint, one of ['public', │
│ 'protected', 'private'] │
│ [default: None] │
│ --accelerator TEXT Accelerator to use. One of ['CPU','GPU'] │
│ [default: None] │
│ --instancetype TEXT [default: None] │
│ --instancesize TEXT [default: None] │
│ --minreplica INTEGER Minimum number of replicas [default: None] │
│ --maxreplica INTEGER Maximum number of replicas [default: None] │
│ --framework TEXT Framework to use [default: huggingface] │
│ --image TEXT Image to use when deploying model endppint. │
│ Must be string representing a valid JSON, │
│ e.g. '{'huggingface': {}}' │
│ [default: None] │
│ --repository TEXT Name of the hf model repository │
│ [default: None] │
│ --revision TEXT Revision of the hf model repository │
│ [default: None] │
│ --task TEXT Task of the model [default: None] │
│ --vendor TEXT Vendor to use. One of ['aws','gcp'] │
│ [default: None] │
│ --region TEXT Vendor specific region, e.g. 'us-east-1' │
│ [default: None] │
│ --overwrite -o Overwrite inpiut file with updated config │
│ --help Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────╯
```
118 changes: 95 additions & 23 deletions docs/endpoint.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,21 +3,23 @@
This page details the usage of the endpoint command

```
Usage: hugie endpoint[OPTIONS] COMMAND[ARGS]...

Options:
--help Show this message and exit.

Commands:
create Create an endpoint
delete Delete an endpoint
info Get info about an endpoint
list List all the deployed endpoints
logs Get logs about an endpoint
test Test and endpoint
update Update an endpoint
Usage: hugie endpoint [OPTIONS] COMMAND [ARGS]...

╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --help Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Commands ───────────────────────────────────────────────────────────────────╮
│ create Create an endpoint │
│ delete Delete an endpoint │
│ info Get info about an endpoint │
│ list List all the deployed endpoints │
│ logs Get logs about an endpoint │
│ test Test an endpoint │
│ update Update an endpoint │
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't this contain config?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it's a separate sub command, so it should be in docs/config

╰──────────────────────────────────────────────────────────────────────────────╯
```


## Create

To create an endpoint:
Expand All @@ -26,34 +28,89 @@ To create an endpoint:
hugie endpoint create examples/development.json
```

## List
Command reference:
```
Usage: hugie endpoint create [OPTIONS] DATA

To list all your endpoints:
Create an endpoint
Args: data (str): Path to JSON data to create the endpoint

╭─ Arguments ──────────────────────────────────────────────────────────────────╮
│ * data TEXT Path JSON data to create the endpoint [default: None] │
│ [required] │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --json Prints the full output in JSON. │
│ --help Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────╯

```
hugie endpoint list

# Use --json option to view all content
## list/ls

List all your endpoints.

Command reference:

hugie endpoint list --json
```
Usage: hugie endpoint list [OPTIONS]

## Update
List all the deployed endpoints

To update an endpoint, edit `examples/development.json`
╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --json Prints the full output in JSON. │
│ --help Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────╯
```


## update

To update an endpoint, edit the config file (e.g. `examples/development.json`), and run:

```
hugie endpoint update examples/development.json
```

Command reference:
```
hugie endpoint update development examples/development.json
Usage: hugie endpoint update [OPTIONS] NAME DATA

Update an endpoint

╭─ Arguments ──────────────────────────────────────────────────────────────────╮
│ * name TEXT Endpoint name [default: None] [required] │
│ * data TEXT Path to JSON data to update the endpoint │
│ [default: None] │
│ [required] │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --json Prints the full output in JSON. │
│ --help Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────╯

```

## Logs

To see the logs:
See the endpoint logs.

Command reference:
```
hugie endpoint logs development
Usage: hugie endpoint logs [OPTIONS] NAME

Get logs about an endpoint

╭─ Arguments ──────────────────────────────────────────────────────────────────╮
│ * name TEXT Endpoint name [default: None] [required] │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --help Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────╯

```


## Delete

To delete the endpoint
Expand All @@ -65,6 +122,21 @@ hugie endpoint delete development
this will ask you if you are sure you want to delete before moving forward. If
you want to force the deletion you can use `--force`

Command reference:
```
Usage: hugie endpoint delete [OPTIONS] NAME

Delete an endpoint

╭─ Arguments ──────────────────────────────────────────────────────────────────╮
│ * name TEXT Endpoint name [default: None] [required] │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --force -f Force deletion without asking user confirmation │
│ --help Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────╯
```

## JSON format

The `endpoint create` and `endpoint update` commands both require JSONs of the fomat:
Expand Down
2 changes: 2 additions & 0 deletions hugie/cli.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
import typer

from hugie.endpoint import app as endpoint_app
from hugie.config import app as config_app

app = typer.Typer()

app.add_typer(endpoint_app, name="endpoint")
app.add_typer(config_app, name="config")

if __name__ == "__main__":
app()
113 changes: 113 additions & 0 deletions hugie/config.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
import srsly
import typer

from hugie.models import InferenceEndpointConfig

app = typer.Typer()


@app.command()
def modify(
path: str,
accountId: str = typer.Option(
None, help="ID of the account (for private endpoints)"
),
name: str = typer.Option(None, help="Name of the endpoint"),
type: str = typer.Option(
None, help="Type of endpoint, one of ['public', 'protected', 'private']"
),
accelerator: str = typer.Option(
"None", help="Accelerator to use. One of ['CPU','GPU']"
),
instanceType: str = typer.Option(None),
instanceSize: str = typer.Option(None),
minReplica: int = typer.Option(None, help="Minimum number of replicas"),
maxReplica: int = typer.Option(None, help="Maximum number of replicas"),
framework: str = typer.Option("huggingface", help="Framework to use"),
image: str = typer.Option(
None,
help="Image to use when deploying model endppint. Must be string representing a valid JSON, e.g. '{'huggingface': {}}'",
),
repository: str = typer.Option(None, help="Name of the hf model repository"),
revision: str = typer.Option(None, help="Revision of the hf model repository"),
task: str = typer.Option(None, help="Task of the model"),
vendor: str = typer.Option(None, help="Vendor to use. One of ['aws','gcp']"),
region: str = typer.Option(None, help="Vendor specific region, e.g. 'us-east-1'"),
overwrite: bool = typer.Option(
False, "--overwrite", "-o", help="Overwrite inpiut file with updated config"
),
):
"""
Modify an existing endpoint config file
"""

config = InferenceEndpointConfig.from_json(path)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does it work with a path that does not exist?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it should fail. I will add tests and error handling for this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Error handling already covered by load_json(). In order to create the file (if it doesn't exist), we would need to have all the other arguments set, otherwise the file would not be properly specified. I would leave this for the current PR and let's discuss about the next one in relation to #54.

Noe that I was also considering adding a hugie config create to create a skeleton config file. Perhaps these two will be related.


# Standard configs

if name:
config.name = name

if type:
config.type = type

if accountId:
config.accountId = accountId

# Model config

if repository:
config.model.repository = repository

if revision:
config.model.revision = revision

if framework:
config.model.framework = framework

if task:
config.model.task = task

if image:
config.model.image = image

# Compute config

if instanceSize:
config.compute.instanceSize = instanceSize

if accelerator:
config.compute.accelerator = accelerator

if instanceType:
config.compute.instanceType = instanceType

# Scaling config

if minReplica:
config.scaling.minReplica = minReplica

if maxReplica:
config.scaling.maxReplica = maxReplica

# Provider config

if vendor:
config.provider.vendor = vendor

if region:
config.provider.region = region

if overwrite:
try:
srsly.write_json(path, config.dict())
typer.secho(f"Updated config at {path}", fg=typer.colors.GREEN)
except Exception:
typer.secho(f"Failed to update config at {path}", fg=typer.colors.RED)

else:
typer.secho(srsly.json_dumps(config.dict()), fg=typer.colors.YELLOW)


if __name__ == "__main__":
app()
Loading