0.64.0

New Features and Improvements

Notebook Integration

ZenML now supports running steps defined in notebook cells with remote orchestrators and step operators. This feature enhances the development workflow by allowing seamless transition from experimentation to production.

• Details: Running remote pipelines from notebooks

Reduced Docker Builds with Code Uploads

We've introduced an option to upload code to the artifact store, enabling Docker build reuse. This feature can significantly speed up iteration, especially when working with remote stacks.

• Default: Enabled
• Configuration: To disable, set DockerSettings.allow_download_from_artifact_store=False
  for steps or pipelines
• Benefits:
  • Faster development cycles
  • No need to register a code repository to reuse builds
  • Builds only occur when requirements or DockerSettings change
• Documentation: Which files are built into the image

AzureML Orchestrator Support

ZenML now supports AzureML as an orchestrator, expanding our list of supported cloud platforms.

• Full Azure Guide: Setting up an Azure stack
• Documentation: AzureML orchestrator

Terraform Modules

We've released new Terraform modules on the Hashicorp registry for provisioning complete MLOps stacks across major cloud providers.

• Features:
  • Automate infrastructure setup for ZenML stack deployment
  • Handle registration of configurations to ZenML server
• More Information: MLOps Terraform ZenML blog post

These updates aim to streamline the MLOps workflow, making it easier to develop, deploy, and manage machine learning pipelines with ZenML.

What's Changed

• Add 0.63.0 to migration testing by @bcdurak in #2893
• Document terraform stack deployment modules by @stefannica in #2898
• README update by @htahir1 in #2901
• Enable Databricks Unity Catalog for MLflow by @safoinme in #2900
• Make urls pop out from the sea of purple/cyan in the logs by @AlexejPenner in #2894
• Add terraform as a supported stack deployment provider by @stefannica in #2902
• Fix Model imports in docs by @strickvl in #2907
• Remove hub references by @schustmi in #2905
• Bump NLP template by @avishniakov in #2912
• Updated step operator docs by @htahir1 in #2908
• Added lightning studio check by @htahir1 in #2910
• Upload code to artifact store by @schustmi in #2895
• AzureML orchestrator by @bcdurak in #2873
• Run steps defined in notebooks with remote orchestrators by @schustmi in #2899
• Fix broken / unparsable md docs file by @strickvl in #2916
• Bump mlflow to 2.15.0 by @christianversloot in #2896
• Remove extra button by @schustmi in #2918
• Added last timestamp to zenserver by @htahir1 in #2913
• A pipeline can't finish successfully in this case by @AlexejPenner in #2903
• Fix the GCP Workload Identity Federation support in the GCP Service Connector by @stefannica in #2914
• Embeddings finetuning guide for LLMOps guide by @strickvl in #2917

🥳 Community Contributions 🥳

We'd like to give a special thanks to @christianversloot who contributed to this release by bumping the mlflow version to 2.15.0

Full Changelog: 0.63.0...0.64.0