Skip to content

Commit

Permalink
Merge branch 'develop' into docs/llms.txt
Browse files Browse the repository at this point in the history
  • Loading branch information
wjayesh committed Jan 6, 2025
2 parents 80a1acb + 2b65e52 commit f1c333d
Show file tree
Hide file tree
Showing 58 changed files with 1,765 additions and 975 deletions.
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,7 @@ Or, through our CLI command:
zenml stack deploy --provider aws
```

Alternatively, if the necessary pieces of infrastructure is already deployed, you can register a cloud stack seamlessly through the stack wizard:
Alternatively, if the necessary pieces of infrastructure are already deployed, you can register a cloud stack seamlessly through the stack wizard:

```bash
zenml stack register <STACK_NAME> --provider aws
Expand Down Expand Up @@ -195,9 +195,9 @@ def trainer(training_df: pd.DataFrame) -> Annotated["model", torch.nn.Module]:

![Exploring ZenML Models](/docs/book/.gitbook/assets/readme_mcp.gif)

### Purpose built for machine learning with integration to you favorite tools
### Purpose built for machine learning with integrations to your favorite tools

While ZenML brings a lot of value of the box, it also integrates into your existing tooling and infrastructure without you having to be locked in.
While ZenML brings a lot of value out of the box, it also integrates into your existing tooling and infrastructure without you having to be locked in.

```python
from bentoml._internal.bento import bento
Expand Down
6 changes: 6 additions & 0 deletions docs/book/component-guide/step-operators/modal.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,12 @@ def my_modal_step():
...
```

{% hint style="info" %}
Note that the `cpu` parameter in `ResourceSettings` currently only accepts a single integer value. This specifies a soft minimum limit - Modal will guarantee at least this many physical cores, but the actual usage could be higher. The CPU cores/hour will also determine the minimum price paid for the compute resources.

For example, with the configuration above (2 CPUs and 32GB memory), the minimum cost would be approximately $1.03 per hour ((0.135 * 2) + (0.024 * 32) = $1.03).
{% endhint %}

This will run `my_modal_step` on a Modal instance with 1 A100 GPU, 2 CPUs, and
32GB of CPU memory.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ def my_pipeline(step_count: int) -> None:
data = load_data_step()
after = []
for i in range(step_count):
train_step(data, learning_rate=i * 0.0001, name=f"train_step_{i}")
train_step(data, learning_rate=i * 0.0001, id=f"train_step_{i}")
after.append(f"train_step_{i}")
model = select_model_step(..., after=after)
```
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,21 @@ zenml code-repository register <NAME> --type=github \

where \<REPOSITORY> is the name of the code repository you are registering, \<OWNER> is the owner of the repository, \<NAME> is the name of the repository, \<GITHUB\_TOKEN> is your GitHub Personal Access Token and \<GITHUB\_URL> is the URL of the GitHub instance which defaults to `https://github.com.` You will need to set a URL if you are using GitHub Enterprise.

{% hint style="warning" %}
Please refer to the section on using secrets for stack configuration in order to securely store your GitHub
Personal Access Token.

```shell
# Using central secrets management
zenml secret create github_secret \
--pa_token=<GITHUB_TOKEN>

# Then reference the username and password
zenml code-repository register ... --token={{github_secret.pa_token}}
...
```
{% endhint %}

After registering the GitHub code repository, ZenML will automatically detect if your source files are being tracked by GitHub and store the commit hash for each pipeline run.

<details>
Expand Down Expand Up @@ -96,6 +111,21 @@ zenml code-repository register <NAME> --type=gitlab \

where `<NAME>` is the name of the code repository you are registering, `<GROUP>` is the group of the project, `<PROJECT>` is the name of the project, \<GITLAB\_TOKEN> is your GitLab Personal Access Token, and \<GITLAB\_URL> is the URL of the GitLab instance which defaults to `https://gitlab.com.` You will need to set a URL if you have a self-hosted GitLab instance.

{% hint style="warning" %}
Please refer to the section on using secrets for stack configuration in order to securely store your GitLab
Personal Access Token.

```shell
# Using central secrets management
zenml secret create gitlab_secret \
--pa_token=<GITLAB_TOKEN>

# Then reference the username and password
zenml code-repository register ... --token={{gitlab_secret.pa_token}}
...
```
{% endhint %}

After registering the GitLab code repository, ZenML will automatically detect if your source files are being tracked by GitLab and store the commit hash for each pipeline run.

<details>
Expand Down
2 changes: 1 addition & 1 deletion scripts/test-migrations.sh
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ else
fi

# List of versions to test
VERSIONS=("0.40.3" "0.43.0" "0.44.3" "0.45.6" "0.47.0" "0.50.0" "0.51.0" "0.52.0" "0.53.1" "0.54.1" "0.55.5" "0.56.4" "0.57.1" "0.60.0" "0.61.0" "0.62.0" "0.63.0" "0.64.0" "0.65.0" "0.68.0" "0.70.0")
VERSIONS=("0.40.3" "0.43.0" "0.44.3" "0.45.6" "0.47.0" "0.50.0" "0.51.0" "0.52.0" "0.53.1" "0.54.1" "0.55.5" "0.56.4" "0.57.1" "0.60.0" "0.61.0" "0.62.0" "0.63.0" "0.64.0" "0.65.0" "0.68.0" "0.70.0" "0.71.0")

# Try to get the latest version using pip index
version=$(pip index versions zenml 2>/dev/null | grep -v YANKED | head -n1 | awk '{print $2}' | tr -d '()')
Expand Down
13 changes: 8 additions & 5 deletions src/zenml/artifacts/artifact_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -104,15 +104,18 @@ def _remove_old_attributes(cls, data: Dict[str, Any]) -> Dict[str, Any]:
)
elif is_model_artifact:
logger.warning(
"`ArtifactConfig.is_model_artifact` is deprecated and will be "
"removed soon. Use `ArtifactConfig.artifact_type` instead."
"`ArtifactConfig(..., is_model_artifact=True)` is deprecated "
"and will be removed soon. Use `ArtifactConfig(..., "
"artifact_type=ArtifactType.MODEL)` instead. For more info: "
"https://docs.zenml.io/user-guide/starter-guide/manage-artifacts"
)
data.setdefault("artifact_type", ArtifactType.MODEL)
elif is_deployment_artifact:
logger.warning(
"`ArtifactConfig.is_deployment_artifact` is deprecated and "
"will be removed soon. Use `ArtifactConfig.artifact_type` "
"instead."
"`ArtifactConfig(..., is_deployment_artifact=True)` is "
"deprecated and will be removed soon. Use `ArtifactConfig(..., "
"artifact_type=ArtifactType.SERVICE)` instead. For more info: "
"https://docs.zenml.io/user-guide/starter-guide/manage-artifacts"
)
data.setdefault("artifact_type", ArtifactType.SERVICE)

Expand Down
4 changes: 3 additions & 1 deletion src/zenml/artifacts/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -414,7 +414,9 @@ def log_artifact_metadata(
"""
logger.warning(
"The `log_artifact_metadata` function is deprecated and will soon be "
"removed. Please use `log_metadata` instead."
"removed. Instead, you can consider using: "
"`log_metadata(metadata={...}, infer_artifact=True, ...)` instead. For more "
"info: https://docs.zenml.io/how-to/model-management-metrics/track-metrics-metadata/attach-metadata-to-an-artifact"
)

from zenml import log_metadata
Expand Down
38 changes: 19 additions & 19 deletions src/zenml/cli/server.py
Original file line number Diff line number Diff line change
Expand Up @@ -587,25 +587,6 @@ def server_list(verbose: bool = False, all: bool = False) -> None:
accessible_pro_servers = client.tenant.list(member_only=not all)
except AuthorizationException as e:
cli_utils.warning(f"ZenML Pro authorization error: {e}")
else:
if not all:
accessible_pro_servers = [
s
for s in accessible_pro_servers
if s.status == TenantStatus.AVAILABLE
]

if not accessible_pro_servers:
cli_utils.declare(
"No ZenML Pro servers that are accessible to the current "
"user could be found."
)
if not all:
cli_utils.declare(
"Hint: use the `--all` flag to show all ZenML servers, "
"including those that the client is not currently "
"authorized to access or are not running."
)

# We update the list of stored ZenML Pro servers with the ones that the
# client is a member of
Expand Down Expand Up @@ -633,6 +614,25 @@ def server_list(verbose: bool = False, all: bool = False) -> None:
stored_server.update_server_info(accessible_server)
pro_servers.append(stored_server)

if not all:
accessible_pro_servers = [
s
for s in accessible_pro_servers
if s.status == TenantStatus.AVAILABLE
]

if not accessible_pro_servers:
cli_utils.declare(
"No ZenML Pro servers that are accessible to the current "
"user could be found."
)
if not all:
cli_utils.declare(
"Hint: use the `--all` flag to show all ZenML servers, "
"including those that the client is not currently "
"authorized to access or are not running."
)

elif pro_servers:
cli_utils.warning(
"The ZenML Pro authentication has expired. Please re-login "
Expand Down
Loading

0 comments on commit f1c333d

Please sign in to comment.