Skip to content
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 50 additions & 0 deletions .github/workflows/scripts.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
name: Scripts

on:
push:
branches: [main]
paths:
- "scripts/**"
- "tests/scripts/**"
- ".github/workflows/scripts.yml"
pull_request:
branches: [main]
paths:
- "scripts/**"
- "tests/scripts/**"
- ".github/workflows/scripts.yml"

concurrency:
group: scripts-${{ github.ref }}
cancel-in-progress: true

permissions: {}

jobs:
test:
name: Test scripts
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: read
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false

- name: Set up Python
uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
with:
python-version: "3.12"

- name: Set up uv
uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2
with:
enable-cache: true

- name: Install dependencies
run: uv sync --all-extras --frozen

- name: Run script tests
run: uv run pytest tests/scripts
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<p align="center">
<img src="https://github.com/microsoft/RAMPART/raw/main/docs/images/RAMPART.svg" alt="RAMPART Logo" width="300"/>
<img src="docs/images/RAMPART.svg" alt="RAMPART Logo" width="300"/>
</p>

<h1 align="center">RAMPART</h1>
Expand Down
68 changes: 40 additions & 28 deletions docs/contributing/release-process.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,36 +34,48 @@ If you find functionality to remove, merge the removal PR to `main` before proce

## 4. Update the Version
Comment thread
nina-msft marked this conversation as resolved.
Outdated

### pyproject.toml
Set the version in `pyproject.toml` to the version established in step 2.
### Git tag
RAMPART derives package versions from Git tags using Hatch VCS and setuptools-scm. No `pyproject.toml` version bump is required for a release. The release version is determined by the `vx.y.z` tag pushed in step 5.

```toml
[project]
name = "RAMPART"
version = "x.y.z"
[tool.hatch.version]
source = "vcs"

[tool.hatch.version.raw-options]
local_scheme = "no-local-version"
```

### Update README File
The README file is published to PyPI and also needs to be updated so the links work properly. _Note: There may not be any links to update, but it is good practice to check in case our README changes._
The `no-local-version` setting omits local version suffixes such as `+g<sha>` because PyPI does not support them for upstream releases. See the [setuptools-scm local scheme documentation](https://setuptools-scm.readthedocs.io/en/latest/extending/#setuptools_scmlocal_scheme) for details.

For development builds on `main` to version correctly, the release tag must be reachable from `main`, meaning it points at a commit that is part of `main`'s history. If it is not, `git describe` finds no tag, setuptools-scm counts commits from the repository root instead, and builds come out as `x.y.devN` versions that sort *before* the release.

Replace all “main” links like “doc/index.md” with “raw” links that have the correct version number, i.e., “https://raw.githubusercontent.com/microsoft/RAMPART/releases/vx.y.z/docs/index.md”.
Tagging the release branch does not satisfy this, because the release branch is never merged into `main`. Cherry-picking the release commit back to `main` does not help either: cherry-pick creates a new commit with a different SHA that the tag does not point to. Instead, tag a commit that is already on `main` and cut the release branch from that tag, as described in step 5.

### Update README File
The README is published to PyPI, so any repository-relative links must resolve for someone reading it there. Because the release is tagged on `main` (step 5), the published README is `main`'s README; there is no separate release-branch copy to maintain.

For images, update using the “raw” link, e.g., “https://raw.githubusercontent.com/microsoft/RAMPART/releases/vx.y.z/docs/images/RAMPART.png”.
Image links can stay relative, e.g., `docs/images/RAMPART.svg`. During package builds, `scripts/hatch_build.py` rewrites those image paths to raw GitHub URLs pinned to the release version.

For directories, update using the “tree” link, e.g., “https://github.com/microsoft/RAMPART/tree/releases/vx.y.z/docs/usage"
If the README gains other repository-relative links (for example to `docs/index.md` or a directory), make them absolute `https://github.com/microsoft/RAMPART/...` URLs on `main`, or extend `scripts/hatch_build.py` to rewrite them at build time the way it already does for images. Do not fix these with a release-only commit on the release branch, because the tag must stay on a commit that is part of `main`.

This is required for the release branch because PyPI does not pick up other files besides the README, which results in local links breaking.
## 5. Tag the Release on `main` and Publish the Release Branch

## 5. Publish the Release Branch to GitHub
Tag the release on `main` first, then cut the release branch from that tag. Tagging `main` rather than the release branch is what keeps the tag reachable from `main`, so development builds version correctly (see the [Git tag](#git-tag) note in step 4).

Commit your changes to a release branch and push the tag:
Confirm any release-prep changes have already merged to `main`, then:

```bash
git checkout -b releases/vx.y.z
git commit -am "release vx.y.z"
git push origin releases/vx.y.z
git checkout main
Comment thread
nina-msft marked this conversation as resolved.
git pull origin main

# Tag the current main commit and push the tag.
git tag -a vx.y.z -m "vx.y.z release"
git push --tags
git push origin vx.y.z

# Cut the release branch from the tagged commit, for release-only
# artifacts and future patch releases.
git checkout -b releases/vx.y.z vx.y.z
git push origin releases/vx.y.z
```


Expand Down Expand Up @@ -120,19 +132,19 @@ Confirm the version matches the release and the package is installed under the e
uv run pytest path/to/RAMPART/tests/integration/test_smoke.py -v
```

If you need to make changes to fix issues found during testing, cherry-pick from `main` after the fix lands:
If you need to make changes to fix issues found during testing, land the fix on `main` first, then move the tag to the new `main` commit so it stays reachable from `main`:

```bash
git checkout main && git pull
git log main # find the commit hash to cherry-pick
git checkout releases/vx.y.z
git cherry-pick <commit-hash>
git push origin releases/vx.y.z
git checkout main && git pull origin main
# After the fix has merged to main:
git tag -a vx.y.z -m "vx.y.z release" --force
git push --tags --force
git push origin vx.y.z --force
# Point the release branch at the retagged commit.
git branch -f releases/vx.y.z vx.y.z
git push origin releases/vx.y.z --force
```

Rebuild the package after any cherry-pick and re-test.
Rebuild the package after re-tagging and re-test.

## 8. Publish to PyPI

Expand All @@ -149,7 +161,7 @@ If successful, the URL `https://pypi.org/project/rampart/x.y.z/` will return the

After the release is on PyPI, open a PR to `main` containing only:

- In line with PyPA [versioning guidance](https://packaging.python.org/en/latest/discussions/versioning/), bump the version in `pyproject.toml` to the next development version (e.g., `x.y.(z+1).dev0` or `x.(y+1).0.dev0`, depending on the next planned release).
- Any follow-up documentation or metadata updates needed after the release. Do not bump the package version in `pyproject.toml`. Because the release was tagged on `main` in step 5, the next commit merged to `main` produces the next development version (for example `x.y.(z+1).devN`) automatically.
- Replace any references to the previous release version in the codebase with the new released version (without `.dev0`) where applicable (e.g., installation docs that pin to the latest tag).

Open this PR from a branch separate from your `releases/vx.y.z` branch.
Expand Down Expand Up @@ -216,10 +228,10 @@ A patch release (e.g., `0.2.0` → `0.2.1`) ships a targeted fix — typically a

Resolve any conflicts manually. Patch-sized fixes typically apply cleanly.

3. **Bump the version** in `pyproject.toml` to the new patch version. Also update any version-pinned links in `README.md`.
3. **Update release-specific references** as needed. Do not bump the package version in `pyproject.toml`; the patch version comes from the `vx.y.z` tag. Also update any version-pinned links in `README.md`.
Comment thread
nina-msft marked this conversation as resolved.
Outdated

```bash
git commit -am "Bump version to x.y.z"
git commit -am "Prepare x.y.z release"
```

4. **Push and tag**:
Expand Down
33 changes: 21 additions & 12 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,12 +1,11 @@
[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"
requires = ["hatchling>=1.30.1", "hatch-vcs>=0.5.0"]
build-backend = "hatchling.build"

[project]
name = "RAMPART"
version = "0.1.1.dev0"
description = "A pytest-native safety testing framework for agentic AI applications"
readme = "README.md"
dynamic = ["readme", "version"]
license = "MIT"
requires-python = ">=3.11"
authors = [
Expand Down Expand Up @@ -43,6 +42,8 @@ onedrive = [

[dependency-groups]
dev = [
"hatch-vcs>=0.5.0",
"hatchling>=1.30.1",
"pre-commit>=4.5.1",
"pytest-cov>=6.1.0",
"pytest-xdist[psutil]>=3.8.0",
Expand All @@ -64,14 +65,17 @@ Issues = "https://github.com/microsoft/RAMPART/issues"
[project.entry-points.pytest11]
rampart = "rampart.pytest_plugin.plugin"

[tool.setuptools.packages.find]
exclude = ["site*", "docs*", "tests*", "scripts*"]
[tool.hatch.metadata.hooks.custom]
path = "scripts/hatch_build.py"

[tool.setuptools.package-data]
rampart = [
"drivers/prompts/*.yaml",
"evaluators/prompts/*.yaml",
]
[tool.hatch.version]
source = "vcs"
Comment thread
nina-msft marked this conversation as resolved.

[tool.hatch.version.raw-options]
local_scheme = "no-local-version"

[tool.hatch.build.targets.wheel]
packages = ["rampart"]

[tool.coverage.run]
source = ["rampart"]
Expand All @@ -83,6 +87,7 @@ show_missing = true
skip_empty = true

[tool.pytest.ini_options]
pythonpath = ["scripts"]
asyncio_mode = "auto"
asyncio_default_fixture_loop_scope = "session"
markers = [
Expand All @@ -105,6 +110,9 @@ extend-ignore = [
]

[tool.ruff.lint.per-file-ignores]
"scripts/hatch_build.py" = [
"INP001", # Top-level build hook
]
"tests/**" = [
"S101", # assert is pytest's API
"D100", "D101", "D102", "D104", "D107", # no docstrings needed
Expand Down Expand Up @@ -141,9 +149,10 @@ max-args = 10

[tool.ty.environment]
python-version = "3.11"
extra-paths = ["scripts"]

[tool.ty.src]
include = ["rampart", "tests"]
include = ["rampart", "tests", "scripts"]

[tool.uv.sources]
pyrit = { git = "https://github.com/microsoft/PyRIT", rev = "6dc8b94139757390286bbce7d53c1f7e58e66e29" } # v0.13.0
87 changes: 87 additions & 0 deletions scripts/hatch_build.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT license.
"""Hatchling metadata hooks for RAMPART package builds."""

from __future__ import annotations

import re
from pathlib import Path

from hatchling.metadata.plugin.interface import MetadataHookInterface

# These patterns only rewrite ``docs/images/`` paths; images stored elsewhere
# (for example ``assets/``) pass through unchanged and must already use an
# absolute URL to render on PyPI.
_GITHUB_IMAGE_URL_PATTERNS = (
Comment thread
nina-msft marked this conversation as resolved.
re.compile(r"(https://github\.com/microsoft/RAMPART/raw/)main(/docs/images/)"),
re.compile(
r"(https://raw\.githubusercontent\.com/microsoft/RAMPART/)main(/docs/images/)",
),
)
_RELATIVE_HTML_IMAGE_URL_PATTERNS = (
re.compile(r'(<img\b[^>]*\bsrc=")(?:\./)?(docs/images/[^"]+)(")'),
re.compile(r"(<img\b[^>]*\bsrc=')(?:\./)?(docs/images/[^']+)(')"),
)
# The URL group stops at whitespace so an optional Markdown title, as in
# ``![alt](docs/images/x.png "title")``, is preserved in group 3 instead of
# being folded into the rewritten image URL.
_RELATIVE_MARKDOWN_IMAGE_URL_PATTERN = re.compile(
r"(!\[[^\]]*\]\()(?:\./)?(docs/images/[^)\s]+)([^)]*\))",
)


def _readme_ref(version: str) -> str:
Comment thread
nina-msft marked this conversation as resolved.
"""Return the Git ref to use for README image URLs."""
if ".dev" in version or "+" in version:
return "main"

return f"v{version}"


def _raw_image_url(*, readme_ref: str, image_path: str) -> str:
"""Return an absolute GitHub raw URL for a README image."""
return (
f"https://raw.githubusercontent.com/microsoft/RAMPART/{readme_ref}/{image_path}"
)


def _render_readme(*, root: Path, version: str) -> str:
"""Return README content rendered for package metadata."""
readme = (root / "README.md").read_text(encoding="utf-8")
readme_ref = _readme_ref(version)

for pattern in _GITHUB_IMAGE_URL_PATTERNS:
readme = pattern.sub(rf"\g<1>{readme_ref}\g<2>", readme)

for pattern in _RELATIVE_HTML_IMAGE_URL_PATTERNS:
readme = pattern.sub(
lambda match: (
f"{match.group(1)}"
f"{_raw_image_url(readme_ref=readme_ref, image_path=match.group(2))}"
f"{match.group(3)}"
),
readme,
)

return _RELATIVE_MARKDOWN_IMAGE_URL_PATTERN.sub(
lambda match: (
f"{match.group(1)}"
f"{_raw_image_url(readme_ref=readme_ref, image_path=match.group(2))}"
f"{match.group(3)}"
),
readme,
)


class ReadmeMetadataHook(MetadataHookInterface):
"""Generate PyPI README metadata with release-pinned image URLs."""

def update(self, metadata: dict[str, object]) -> None:
"""Update project metadata in-place."""
metadata["readme"] = {
"content-type": "text/markdown",
"text": _render_readme(
root=Path(self.root),
version=str(metadata["version"]),
),
}
2 changes: 2 additions & 0 deletions tests/scripts/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT license.
Loading
Loading