Add config requirements and environment variables (and fix broken links!) (#238)

* Sync config requirements with updated requirements.txt

Remove openai dependency and add missing dependencies (zenml, exa-py, requests, anthropic) to match the updated requirements.txt file across all config files.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>

* Add environment variables to config files and update README

- Add docker environment variable placeholders to all config files for OPENROUTER_API_KEY, TAVILY_API_KEY, EXA_API_KEY, ANTHROPIC_API_KEY, and Langfuse credentials
- Add cloud orchestrator configuration section to README explaining how to replace placeholder values with actual API keys for cloud deployments
- Include security recommendation to use cloud provider secret management services

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>

* Fix typo

* Fix broken links

* Replace hardcoded API key placeholders with environment variable substitution

- Update all config files to use ${VARIABLE_NAME} syntax instead of "YOUR_VARIABLE_NAME" placeholders
- Update README to reflect automatic environment variable pickup
- Users no longer need to manually edit config files for cloud deployment

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>

* Remove redundant zenml requirement from config files

ZenML is the orchestration framework already running the pipeline,
so it doesn't need to be installed as a dependency inside Docker containers.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>

---------

Co-authored-by: Claude <[email protected]>

Author: strickvl

credit-scorer/docs/guides/cloud_deployment.md

@@ -1,6 +1,6 @@
 ## ☁️ Cloud Deployment
 
-CreditScorer supports storing artifacts remotely and executing pipelines on cloud infrastructure. For this example, we'll use AWS, but you can use any cloud provider you want. You can also refer to the [AWS Integration Guide](https://docs.zenml.io/how-to/popular-integrations/aws-guide) for detailed instructions.
+CreditScorer supports storing artifacts remotely and executing pipelines on cloud infrastructure. For this example, we'll use AWS, but you can use any cloud provider you want. You can also refer to the [AWS Integration Guide](https://docs.zenml.io/stacks/popular-stacks/aws-guide) for detailed instructions.
 
 ### AWS Setup
 
@@ -75,6 +75,6 @@ Similar setup processes can be followed for other cloud providers:
 
 For detailed configuration options for these providers, refer to the ZenML documentation:
 
-- [GCP Integration Guide](https://docs.zenml.io/how-to/popular-integrations/gcp-guide)
-- [Azure Integration Guide](https://docs.zenml.io/how-to/popular-integrations/azure-guide)
-- [Kubernetes Integration Guide](https://docs.zenml.io/how-to/popular-integrations/kubernetes)
+- [GCP Integration Guide](https://docs.zenml.io/stacks/popular-stacks/gcp-guide)
+- [Azure Integration Guide](https://docs.zenml.io/stacks/popular-stacks/azure-guide)
+- [Kubernetes Integration Guide](https://docs.zenml.io/stacks/popular-stacks/kubernetes)

databricks-production-qa-demo/steps/deployment/deployment_deploy.py

@@ -45,7 +45,7 @@ def deployment_deploy() -> Annotated[
     In this example, the step can be configured to use different input data.
     See the documentation for more information:
 
-        https://docs.zenml.io/how-to/pipeline-development/use-configuration-files
+        https://docs.zenml.io/concepts/steps_and_pipelines/yaml_configuration
 
     Args:
         dataset_inf: The inference dataset.

databricks-production-qa-demo/steps/etl/data_loader.py

@@ -44,7 +44,7 @@ def data_loader(
     In this example, the step can be configured with number of rows and logic
     to drop target column or not. See the documentation for more information:
 
-        https://docs.zenml.io/how-to/build-pipelines/use-pipeline-step-parameters
+        https://docs.zenml.io/concepts/steps_and_pipelines#pipeline-parameterization
 
     Args:
         is_inference: If `True` subset will be returned and target column

databricks-production-qa-demo/steps/etl/train_data_preprocessor.py

@@ -52,7 +52,7 @@ def train_data_preprocessor(
     columns and normalize numerical columns. See the documentation for more
     information:
 
-        https://docs.zenml.io/how-to/build-pipelines/use-pipeline-step-parameters
+        https://docs.zenml.io/concepts/steps_and_pipelines#pipeline-parameterization
 
     Args:
         dataset_trn: The train dataset.

databricks-production-qa-demo/steps/etl/train_data_splitter.py

@@ -41,7 +41,7 @@ def train_data_splitter(
     In this example, the step can be configured to use different test
     set sizes. See the documentation for more information:
 
-        https://docs.zenml.io/how-to/pipeline-development/use-configuration-files
+        https://docs.zenml.io/concepts/steps_and_pipelines/yaml_configuration
 
     Args:
         dataset: Dataset read from source.

databricks-production-qa-demo/steps/hp_tuning/hp_tuning_single_search.py

@@ -50,7 +50,7 @@ def hp_tuning_single_search(
     to use different input datasets and also have a flag to fall back to default
     model architecture. See the documentation for more information:
 
-        https://docs.zenml.io/how-to/pipeline-development/use-configuration-files
+        https://docs.zenml.io/concepts/steps_and_pipelines/yaml_configuration
 
     Args:
         model_package: The package containing the model to use for hyperparameter tuning.

databricks-production-qa-demo/steps/inference/inference_predict.py

@@ -43,7 +43,7 @@ def inference_predict(
     In this example, the step can be configured to use different input data.
     See the documentation for more information:
 
-        https://docs.zenml.io/how-to/pipeline-development/use-configuration-files
+        https://docs.zenml.io/concepts/steps_and_pipelines/yaml_configuration
 
     Args:
         dataset_inf: The inference dataset.

databricks-production-qa-demo/steps/promotion/compute_performance_metrics_on_current_data.py

@@ -44,7 +44,7 @@ def compute_performance_metrics_on_current_data(
     and target environment stage for promotion.
     See the documentation for more information:
 
-        https://docs.zenml.io/how-to/pipeline-development/use-configuration-files
+        https://docs.zenml.io/concepts/steps_and_pipelines/yaml_configuration
 
     Args:
         dataset_tst: The test dataset.

databricks-production-qa-demo/steps/promotion/promote_with_metric_compare.py

@@ -46,7 +46,7 @@ def promote_with_metric_compare(
     and target environment stage for promotion.
     See the documentation for more information:
 
-        https://docs.zenml.io/how-to/pipeline-development/use-configuration-files
+        https://docs.zenml.io/concepts/steps_and_pipelines/yaml_configuration
 
     Args:
         latest_metric: Recently trained model metric results.

databricks-production-qa-demo/steps/training/model_trainer.py

@@ -72,7 +72,7 @@ def model_trainer(
     hyperparameters to the model constructor. See the documentation for more
     information:
 
-        https://docs.zenml.io/how-to/pipeline-development/use-configuration-files
+        https://docs.zenml.io/concepts/steps_and_pipelines/yaml_configuration
 
     Args:
         dataset_trn: The preprocessed train dataset.