Add experiment comparison tool docs (#3287)

* Add new visualizations and enhance documentation for tracking metrics

* Update image paths in README for Experiment Comparison tool documentation

* Update README.md to improve formatting of image displays in Experiment Comparison tool documentation

* Update README.md to include links to ZenML Pro dashboard for enhanced clarity in Experiment Comparison tool documentation

* Add documentation for Experiment Comparison tool in ZenML Pro dashboard for starter guide

- Introduced a new section on comparing metadata across runs, detailing the features and benefits of the Experiment Comparison tool.
- Added descriptions for Table View and Parallel Coordinates View, including their functionalities and use cases.
- Provided step-by-step instructions for accessing the comparison tool and sharing configurations.
- Included warnings about the feature being in Alpha Preview and encouraged user feedback.

* Update manage-artifacts.md to include a hyperlink to the ZenML Pro dashboard in the Experiment Comparison tool section for improved accessibility and clarity.

* Refactor section headers in manage-artifacts.md for improved clarity and consistency. Changed headings from '###' to '##' for artifact type specification, consuming external artifacts, and consuming artifacts produced by other pipelines.

* Add Experiment Comparison video asset and update README.md for enhanced documentation

- Introduced a new image asset for the Experiment Comparison video to improve visual guidance.
- Updated the README.md to replace the existing iframe with a markdown image link for better accessibility and formatting.

* Update docs/book/how-to/model-management-metrics/track-metrics-metadata/README.md

docs/book/how-to/model-management-metrics/track-metrics-metadata/README.md

@@ -1,15 +1,13 @@
 ---
 icon: ufo-beam
-description: Tracking metrics and metadata
+description: Tracking and comparing metrics and metadata
 ---
 
 # Track metrics and metadata
 
-ZenML provides a unified way to log and manage metrics and metadata through 
-the `log_metadata` function. This versatile function allows you to log 
-metadata across various entities like models, artifacts, steps, and runs 
-through a single interface. Additionally, you can adjust if you want to 
-automatically the same metadata for the related entities.
+ZenML provides a unified way to log and manage metrics and metadata through the `log_metadata` function. This versatile function allows you to log metadata across various entities like models, artifacts, steps, and runs through a single interface. Additionally, you can adjust if you want to automatically log the same metadata for related entities.
+
+## Logging Metadata
 
 ### The most basic use-case
 
@@ -24,25 +22,89 @@ def my_step() -> ...:
     ...
 ```
 
-This will log the `accuracy` for the step, its pipeline run, and if provided 
-its model version.
+This will log the `accuracy` for the step, its pipeline run, and if provided its model version.
+
+### A real-world example
+
+Here's a more comprehensive example showing how to log various types of metadata in a machine learning pipeline:
+
+```python
+from zenml import step, pipeline, log_metadata
+
+@step
+def process_engine_metrics() -> float:
+    # does some machine learning things
+
+    # Log operational metrics
+    log_metadata(
+        metadata={
+            "engine_temperature": 3650,  # Kelvin
+            "fuel_consumption_rate": 245,  # kg/s
+            "thrust_efficiency": 0.92,
+        }
+    )
+    return 0.92
+
+@step
+def analyze_flight_telemetry(efficiency: float) -> None:
+    # does some more machine learning things
+
+    # Log performance metrics
+    log_metadata(
+        metadata={
+            "altitude": 220000,  # meters
+            "velocity": 7800,  # m/s
+            "fuel_remaining": 2150,  # kg
+            "mission_success_prob": 0.9985,
+        }
+    )
+
+@pipeline
+def telemetry_pipeline():
+    efficiency = process_engine_metrics()
+    analyze_flight_telemetry(efficiency)
+```
+
+This data can be visualized and compared in the ZenML Pro dashboard. The
+illustrations below show the data from this example in the [ZenML Pro](https://www.zenml.io/pro) dashboard
+using the Experiment Comparison tool.
+
+{% hint style="warning" %}
+This feature is currently in Alpha Preview. We encourage you to share feedback about your use cases and requirements through our Slack community.
+{% endhint %}
+
+## Visualizing and Comparing Metadata (Pro)
+
+Once you've logged metadata in your pipelines, you can use ZenML's Experiment Comparison tool to analyze and compare metrics across different runs. This feature is available in the [ZenML Pro](https://www.zenml.io/pro) dashboard.
+
+[![Experiment Comparison Introduction Video](../../../../book/.gitbook/assets/experiment_comparison_video.png)](https://www.loom.com/share/693b2d829600492da7cd429766aeba6a?sid=7182e55b-31e9-4b38-a3be-07c989dbea32)
+
+### Comparison Views
+
+The Experiment Comparison tool offers two complementary views for analyzing your pipeline metadata:
+
+1. **Table View**: Compare metadata across runs with automatic change tracking
+
+![Table View](../../../../book/.gitbook/assets/table-view.png)
+
+2. **Parallel Coordinates Plot**: Visualize relationships between different metrics
+
+![Parallel Coordinates](../../../../book/.gitbook/assets/coordinates-view.png)
+
+The tool lets you compare up to 20 pipeline runs simultaneously and supports any
+numerical metadata (`float` or `int`) that you've logged in your pipelines.
 
 ### Additional use-cases
 
-The `log_metadata` function also supports various use-cases by allowing you to 
-specify the target entity (e.g., model, artifact, step, or run) with flexible 
-parameters. You can learn more about these use-cases in the following pages:
+The `log_metadata` function supports various use-cases by allowing you to specify the target entity (e.g., model, artifact, step, or run) with flexible parameters. You can learn more about these use-cases in the following pages:
 
 - [Log metadata to a step](attach-metadata-to-a-step.md)
 - [Log metadata to a run](attach-metadata-to-a-run.md)
 - [Log metadata to an artifact](attach-metadata-to-an-artifact.md)
 - [Log metadata to a model](attach-metadata-to-a-model.md)
 
 {% hint style="warning" %}
-The older methods for logging metadata to specific entities, such as 
-`log_model_metadata`, `log_artifact_metadata`, and `log_step_metadata`, are 
-now deprecated. It is recommended to use `log_metadata` for all future 
-implementations.
+The older methods for logging metadata to specific entities, such as `log_model_metadata`, `log_artifact_metadata`, and `log_step_metadata`, are now deprecated. It is recommended to use `log_metadata` for all future implementations.
 {% endhint %}
 
 <figure><img src="https://static.scarf.sh/a.png?x-pxid=f0b4f458-0a54-4fcd-aa95-d5ee424815bc" alt="ZenML Scarf"><figcaption></figcaption></figure>

docs/book/user-guide/starter-guide/manage-artifacts.md

@@ -167,7 +167,58 @@ def annotation_approach() -> (
     return "string"
 ```
 
-### Specify a type for your artifacts
+## Comparing metadata across runs (Pro)
+
+The [ZenML Pro](https://www.zenml.io/pro) dashboard includes an Experiment Comparison tool that allows you to visualize and analyze metadata across different pipeline runs. This feature helps you understand patterns and changes in your pipeline's behavior over time.
+
+### Using the comparison views
+
+The tool offers two complementary views for analyzing your metadata:
+
+#### Table View
+The tabular view provides a structured comparison of metadata across runs:
+
+![Comparing metadata values across different pipeline runs in table view.](../../../book/.gitbook/assets/table-view.png)
+
+This view automatically calculates changes between runs and allows you to:
+
+* Sort and filter metadata values
+* Track changes over time
+* Compare up to 20 runs simultaneously
+
+#### Parallel Coordinates View
+The parallel coordinates visualization helps identify relationships between different metadata parameters:
+
+![Comparing metadata values across different pipeline runs in parallel coordinates view.](../../../book/.gitbook/assets/coordinates-view.png)
+
+This view is particularly useful for:
+
+* Discovering correlations between different metrics
+* Identifying patterns across pipeline runs
+* Filtering and focusing on specific parameter ranges
+
+### Accessing the comparison tool
+
+To compare metadata across runs:
+
+1. Navigate to any pipeline in your dashboard
+2. Click the "Compare" button in the top navigation
+3. Select the runs you want to compare
+4. Switch between table and parallel coordinates views using the tabs
+
+{% hint style="info" %}
+The comparison tool works with any numerical metadata (`float` or `int`) that you've logged in your pipelines. Make sure to log meaningful metrics in your steps to make the most of this feature.
+{% endhint %}
+
+### Sharing comparisons
+
+The tool preserves your comparison configuration in the URL, making it easy to share specific views with team members. Simply copy and share the URL to allow others to see the same comparison with identical settings and filters.
+
+{% hint style="warning" %}
+This feature is currently in Alpha Preview. We encourage you to share feedback about your use cases and requirements through our Slack community.
+{% endhint %}
+
+## Specify a type for your artifacts
 
 Assigning a type to an artifact allows ZenML to highlight them differently in the dashboard and also lets you filter your artifacts better.
 
@@ -193,7 +244,7 @@ model = ...
 save_artifact(model, name="model", artifact_type=ArtifactType.MODEL)
 ```
 
-### Consuming external artifacts within a pipeline
+## Consuming external artifacts within a pipeline
 
 While most pipelines start with a step that produces an artifact, it is often the case to want to consume artifacts external from the pipeline. The `ExternalArtifact` class can be used to initialize an artifact within ZenML with any arbitrary data type.
 
@@ -226,7 +277,7 @@ Optionally, you can configure the `ExternalArtifact` to use a custom [materializ
 Using an `ExternalArtifact` for your step automatically disables caching for the step.
 {% endhint %}
 
-### Consuming artifacts produced by other pipelines
+## Consuming artifacts produced by other pipelines
 
 It is also common to consume an artifact downstream after producing it in an upstream pipeline or step. As we have learned in the [previous section](../../how-to/pipeline-development/build-pipelines/fetching-pipelines.md#fetching-artifacts-directly), the `Client` can be used to fetch artifacts directly inside the pipeline code: