Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 13 additions & 23 deletions .github/workflows/cdm-ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,36 +13,26 @@ jobs:
changes:
runs-on: [ self-hosted, workflow-overhead ]
outputs:
real-ci: ${{ steps.filter.outputs.real-ci }}
faux-ci: ${{ steps.filter.outputs.faux-ci }}
only-docs: ${{ steps.filter.outputs.only_changed }}
steps:
- uses: actions/checkout@v4
- uses: dorny/paths-filter@v3
- uses: tj-actions/changed-files@v47
id: filter
with:
filters: |
real-ci:
- '**'
- '!LICENSE'
- '!**.md'
- '!.github/rulesets/**'
- '!.github/workflows/run-crucible-tracking.yaml'
- '!.github/workflows/crucible-ci.yaml'
- '!.github/workflows/cdm-ci.yaml'
- '!docs/**'
faux-ci:
- 'LICENSE'
- '**.md'
- '.github/rulesets/**'
- '.github/workflows/run-crucible-tracking.yaml'
- '.github/workflows/crucible-ci.yaml'
- '.github/workflows/cdm-ci.yaml'
- 'docs/**'
files: |
LICENSE
*.md
**/*.md
.github/rulesets/**
.github/workflows/run-crucible-tracking.yaml
.github/workflows/crucible-ci.yaml
.github/workflows/cdm-ci.yaml
docs/**

real-cdm-ci:
runs-on: ubuntu-latest
needs: changes
if: ${{ needs.changes.outputs.real-ci == 'true' }}
if: ${{ needs.changes.outputs.only-docs != 'true' }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v3
Expand All @@ -59,7 +49,7 @@ jobs:
faux-cdm-ci:
runs-on: [ self-hosted, workflow-overhead ]
needs: changes
if: ${{ needs.changes.outputs.faux-ci == 'true' && needs.changes.outputs.real-ci == 'false' }}
if: ${{ needs.changes.outputs.only-docs == 'true' }}
steps:
- run: 'echo "faux-cdm-ci-complete"'

Expand Down
36 changes: 13 additions & 23 deletions .github/workflows/crucible-ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,35 +9,25 @@ jobs:
changes:
runs-on: [ self-hosted, workflow-overhead ]
outputs:
real-ci: ${{ steps.filter.outputs.real-ci }}
faux-ci: ${{ steps.filter.outputs.faux-ci }}
only-docs: ${{ steps.filter.outputs.only_changed }}
steps:
- uses: actions/checkout@v4
- uses: dorny/paths-filter@v3
- uses: tj-actions/changed-files@v47
id: filter
with:
filters: |
real-ci:
- '**'
- '!LICENSE'
- '!**.md'
- '!.github/rulesets/**'
- '!.github/workflows/run-crucible-tracking.yaml'
- '!.githuw/workflows/crucible-ci.yaml'
- '!.githuw/workflows/cdm-ci.yaml'
- '!docs/**'
faux-ci:
- 'LICENSE'
- '**.md'
- '.github/rulesets/**'
- '.github/workflows/run-crucible-tracking.yaml'
- '.github/workflows/crucible-ci.yaml'
- '.github/workflows/cdm-ci.yaml'
- 'docs/**'
files: |
LICENSE
*.md
**/*.md
.github/rulesets/**
.github/workflows/run-crucible-tracking.yaml
.github/workflows/crucible-ci.yaml
.github/workflows/cdm-ci.yaml
docs/**

call-real-core-crucible-ci:
needs: changes
if: ${{ needs.changes.outputs.real-ci == 'true' }}
if: ${{ needs.changes.outputs.only-docs != 'true' }}
uses: perftool-incubator/crucible-ci/.github/workflows/core-crucible-ci.yaml@main
with:
ci_target: "CommonDataModel"
Expand All @@ -50,7 +40,7 @@ jobs:

call-faux-core-crucible-ci:
needs: changes
if: ${{ needs.changes.outputs.faux-ci == 'true' && needs.changes.outputs.real-ci == 'false' }}
if: ${{ needs.changes.outputs.only-docs == 'true' }}
uses: perftool-incubator/crucible-ci/.github/workflows/faux-core-crucible-ci.yaml@main

crucible-ci-complete:
Expand Down
46 changes: 46 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# CommonDataModel (CDM) - Performance Data Model

## Purpose
Defines a unified data model for storing and querying performance test data in OpenSearch. Provides index templates, query tools, and an HTTP query server.

## Languages
- **JavaScript/Node.js**: Query library and server (`queries/cdmq/`)
- **Bash**: Template management scripts (`templates/`)

## Key Directories
| Path | Purpose |
|------|---------|
| `queries/cdmq/` | Node.js query library and HTTP server |
| `templates/` | OpenSearch index templates and management scripts |
| `workflows/` | Documentation (result-calculation methodology) |

## Key Files in `queries/cdmq/`
| File | Purpose |
|------|---------|
| `cdm.js` | Core query library — document CRUD, search, aggregation |
| `server.js` | HTTP server exposing CDM queries as REST endpoints |
| `add-run.js` | Indexes a complete benchmark run |
| `delete-run.js` | Removes a run from the index |
| `get-metric-data.js` | Retrieves metric data for a run |
| `get-result-summary.js` | Generates result summaries |
| `get-primary-periods.js` | Retrieves primary measurement periods |
| `package.json` | Node.js dependencies |

## Data Model Hierarchy
`run` > `iteration` > `sample` > `period` > `metric_desc` + `metric_data`

Supporting document types: `param`, `tag`, `config_*`

## Versioning
- Versions tracked as git branches and in `VERSION` file (currently `v8dev`)
- `cdm.js` exports `supportedCdmVersions` array: `['v7dev', 'v8dev', 'v9dev']`
- Index naming pattern: `cdm{VERSION}-{DOCTYPE}*` (e.g., `cdmv8dev-metric_data*`)

## Templates (`templates/`)
- `.base` files define index mappings for each document type
- `build.sh` / `Makefile` generate actual template commands
- `init.sh` initializes the OpenSearch indices

## Code Style
- JavaScript: Prettier formatting enforced (2-space indent, checked in CI via `cdm-ci.yaml`)
- Bash: Standard 4-space indentation with vim/emacs modelines
29 changes: 18 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,25 +1,32 @@
# CommonDataModel
A reference data model that uses elasticsearch to unify data for monitoring and benchmarking
A reference data model that uses OpenSearch to unify data for monitoring and benchmarking
## Introduction
### What's The Problem?
Our Data is the problem. When it comes to performance benchmarking, or just monitoring performance, we have a data compatibility problem. We don't have a good standard to describe our environment and include performance characterization together. When we don't have this, many of the solutions created to help us visualize, investigate, and identify performance issues are incompatible with each other. Solutions are often designed with very specific environments and cannot be reused for other environments.
### How Can We Fix This?
We can define a common way to store information about our environment, our performance tests (if any), and metrics and events we collect. Having a common way to process this information allows us to query, summarize, and visualize performance data across many situations, from comparing compiler performance to identifying bottlenecks in large cloud deployments.
### What This Project Will Include
We want to provide enough information so that anyone can start storing and querying this data in a common way. We aim to provide the following:
* elasticsearch index templates
* elasticsearch query scripts
### What This Project Will Not Include
### What This Project Includes
We provide enough information so that anyone can start storing and querying this data in a common way:
* OpenSearch index templates
* A Node.js query library and HTTP query server (`queries/cdmq/`)
### What This Project Does Not Include
* data conversion scripts: conversion of data is expected to be in the other projects, for example: [uperf-post-process](https://github.com/perftool-incubator/bench-uperf/blob/master/uperf-post-process)
* data indexing scripts: indexing of data is also expected to be in other projects, for example: [rickshaw-index](https://github.com/perftool-incubator/rickshaw/blob/master/rickshaw-index)
## Directory/Layout
[./templates](./templates)
This is where all ES templates will reside, ready for emitting to ES
OpenSearch index templates and management scripts (init, build, delete).

[./queries](./queries)
One or more query implementations may be found here. All query implementations use the same fundamental queries to get data, but may differ in output, features, and programming language.
The `cdmq` query implementation, built on Node.js. Includes a core query library (`cdm.js`), command-line query scripts, and an HTTP server (`server.js`) that exposes CDM queries as REST endpoints.

[./workflows](./workflows)
Documentation for result calculation methodology.

[./VERSION](./VERSION)
The current CDM schema version (e.g., `v8dev`).
## Versioning
The common data model will be versioned, and for each version, the number of document-types and their field-names may change. In general, newer versions will attempt to include all document-types and field-names of previous versions. The version number is represented in whole numbers, with a git branch for each. Once a new version is established, only minor fixes should be applied to that version, with no changes to the schema. If there is a major problem with a version, it should be marked as non-functional.
The common data model will be versioned, and for each version, the number of document-types and their field-names may change. In general, newer versions will attempt to include all document-types and field-names of previous versions. The current version is tracked in the `VERSION` file. Once a new version is established, only minor fixes should be applied to that version, with no changes to the schema. If there is a major problem with a version, it should be marked as non-functional.
## Using With Other Projects
This project is not really intended to be used standalone (however it is possible). In most cases, this project should be incorporated into a larger suite of automation for benchmarking and/or reporting, like [crucible](https://github.com/perftool-incubator/crucible)
This project is not really intended to be used standalone (however it is possible). In most cases, this project should be incorporated into a larger suite of automation for benchmarking and/or reporting, like [crucible](https://github.com/perftool-incubator/crucible).
## Project Status
The project is still under heavy development, but minimally viable. The templates and scripts are actively leveraged in the [crucible](https://github.com/perftool-incubator/crucible) project, for example, [here](https://github.com/perftool-incubator/crucible/blob/master/config/init-es.sh) and [here](https://github.com/perftool-incubator/crucible/blob/c8ca954aaf1cd0dd7f28c5b4167cb5708e32c3d9/bin/_main#L235)
The templates and query tools are actively leveraged in the [crucible](https://github.com/perftool-incubator/crucible) project.
Loading