Evals Tutorials: removing starter toolkit; including CLI examples

[BharathiSrini]

Amazon Bedrock AgentCore Samples Pull Request

Important

1. We strictly follow a issue-first approach, please first open an issue relating to this Pull Request.
2. Once this Pull Request is ready for review please attach review ready label to it. Only PRs with review ready will be reviewed.

Issue number:

Concise description of the PR

Changes to Evals tutorials, because of starter-toolkit deprecation and introducing agentcore-cli.
CLI examples in README
Separate deploy.py scripts using the Runtime SDK
This script is used in notebooks
Notebooks only contain SDK references and no CLI

User experience

Please share what the user experience looks like before and after this change

Checklist

If your change doesn't seem to apply, please leave them unchecked.

• [x ] I have reviewed the contributing guidelines
• [ x] Add your name to CONTRIBUTORS.md
• [ x] Have you checked to ensure there aren't other open Pull Requests for the same update/change?
• Are you uploading a dataset?
• [x ] Have you documented Introduction, Architecture Diagram, Prerequisites, Usage, Sample Prompts, and Clean Up steps in your example README?
• [x ] I agree to resolve any issues created for this example in the future.
• [ x] I have performed a self-review of this change
• [ x] Changes have been tested
• [ x] Changes are documented

Acknowledgment

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of the project license.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[github-actions]

Latest scan for commit: fb2f9df | Updated: 2026-05-05 23:23:23 UTC

Security Scan Results

Scan Metadata

• Project: ASH
• Scan executed: 2026-05-05T23:23:10+00:00
• ASH version: 3.0.0

Summary

Scanner Results

The table below shows findings by scanner, with status based on severity thresholds and dependencies:

Column Explanations:

Severity Levels (S/C/H/M/L/I):

• Suppressed (S): Security findings that have been explicitly suppressed/ignored and don't affect the scanner's pass/fail status
• Critical (C): The most severe security vulnerabilities requiring immediate remediation (e.g., SQL injection, remote code execution)
• High (H): Serious security vulnerabilities that should be addressed promptly (e.g., authentication bypasses, privilege escalation)
• Medium (M): Moderate security risks that should be addressed in normal development cycles (e.g., weak encryption, input validation issues)
• Low (L): Minor security concerns with limited impact (e.g., information disclosure, weak recommendations)
• Info (I): Informational findings for awareness with minimal security risk (e.g., code quality suggestions, best practice recommendations)

Other Columns:

• Time: Duration taken by each scanner to complete its analysis
• Action: Total number of actionable findings at or above the configured severity threshold that require attention

Scanner Results:

• PASSED: Scanner found no security issues at or above the configured severity threshold - code is clean for this scanner
• FAILED: Scanner found security vulnerabilities at or above the threshold that require attention and remediation
• MISSING: Scanner could not run because required dependencies/tools are not installed or available
• SKIPPED: Scanner was intentionally disabled or excluded from this scan
• ERROR: Scanner encountered an execution error and could not complete successfully

Severity Thresholds (Thresh Column):

• CRITICAL: Only Critical severity findings cause scanner to fail
• HIGH: High and Critical severity findings cause scanner to fail
• MEDIUM (MED): Medium, High, and Critical severity findings cause scanner to fail
• LOW: Low, Medium, High, and Critical severity findings cause scanner to fail
• ALL: Any finding of any severity level causes scanner to fail

Threshold Source: Values in parentheses indicate where the threshold is configured:

• (g) = global: Set in the global_settings section of ASH configuration
• (c) = config: Set in the individual scanner configuration section
• (s) = scanner: Default threshold built into the scanner itself

Statistics calculation:

• All statistics are calculated from the final aggregated SARIF report
• Suppressed findings are counted separately and do not contribute to actionable findings
• Scanner status is determined by comparing actionable findings to the threshold

Scanner	S	C	H	M	L	I	Time	Action	Result	Thresh
bandit	0	4	0	0	6	0	1.6s	4	FAILED	MED (g)
cdk-nag	0	0	0	0	0	0	28.8s	0	PASSED	MED (g)
cfn-nag	0	0	0	0	0	0	43ms	0	PASSED	MED (g)
checkov	0	0	0	0	0	0	5.4s	0	PASSED	MED (g)
detect-secrets	0	0	0	0	0	0	1.3s	0	PASSED	MED (g)
grype	0	0	0	0	0	0	41.1s	0	PASSED	MED (g)
npm-audit	0	0	0	0	0	0	265ms	0	PASSED	MED (g)
opengrep	0	0	0	0	0	0	<1ms	0	SKIPPED	MED (g)
semgrep	0	0	0	0	0	0	<1ms	0	MISSING	MED (g)
syft	0	0	0	0	0	0	2.1s	0	PASSED	MED (g)

Detailed Findings

Show 4 actionable findings

Finding 1: B108

• Severity: HIGH
• Scanner: bandit
• Rule ID: B108
• Location: 01-tutorials/07-AgentCore-evaluations/00-prereqs/deploy_agents.py:165-167

Description:
Probable insecure usage of temp file/directory.

Code Snippet:

"""Package dependencies + agent code, upload to S3, return (bucket, key)."""
    build_dir = Path(f"/tmp/{name}_build")
    if build_dir.exists():

---

Finding 2: B307

• Severity: HIGH
• Scanner: bandit
• Rule ID: B307
• Location: 01-tutorials/07-AgentCore-evaluations/00-prereqs/eval_agent_langgraph.py:55-57

Description:
Use of possibly insecure function - consider using safer ast.literal_eval.

Code Snippet:

# Evaluate the expression safely
        result = eval(expression, safe_dict)
        return str(result)

---

Finding 3: B108

• Severity: HIGH
• Scanner: bandit
• Rule ID: B108
• Location: 01-tutorials/07-AgentCore-evaluations/03-evaluation-workflows/02-simulating-agent-interactions/deploy_shopping_concierge_agent.py:43-45

Description:
Probable insecure usage of temp file/directory.

Code Snippet:

_S3_KEY = f"{_AGENT_NAME}/deployment_package.zip"
_BUILD_DIR = Path(f"/tmp/{_AGENT_NAME}_build")

---

Finding 4: B108

• Severity: HIGH
• Scanner: bandit
• Rule ID: B108
• Location: 01-tutorials/07-AgentCore-evaluations/05-groundtruth-based-evalautions/deploy_hr_assistant_agent.py:42-44

Description:
Probable insecure usage of temp file/directory.

Code Snippet:

_S3_KEY = f"{_AGENT_NAME}/deployment_package.zip"
_BUILD_DIR = Path(f"/tmp/{_AGENT_NAME}_build")

---

Report generated by Automated Security Helper (ASH) at 2026-05-05T23:23:04+00:00