Skip to content

Commit

Permalink
Update Docs for Trails (#132)
Browse files Browse the repository at this point in the history
* Remove hidden:true flag from attest leaf commands and use hidden flag in root attest command

* Start updating tutorial docs for Trails

* Update name of cyber-dojo microservice from shas to version-reporter

* Fix 'kosli create environment' command in tutorials/simulating_a_devops_system.md

* Fix content of tutorials/simulating_a_devops_system.md to match actual output

* Fix content of tutorials/tracing_a_production_incident_back_to_git_commits.md to match actual output

* Split create-flow and create-trail into separate sections in tutorials/get_familiar_with_Kosli.md

* More updates to tutorials/get_familiar_with_Kosli.md

* Add comment to tutorials/get_familiar_with_Kosli.md

* Start to update top-level overview pages

* Add comment to md files re descriptions of what a Flow is needing to live at top-level only

* Start adding overview text at top of what-is-kosli file

* Trigger deploy of branch on netlify

* Add workflow for deployment of branch docs

* Add workflow for branch deployment of docs

* edit docs branch deploy workflow

* Only use link checker on production branch

* Update netlify config to ignore branch previews

* Revert "Add workflow for branch deployment of docs"

This reverts commit 72e6446.

* Restore github token in workflow

* Restore property in workflow

* Document deployment of staging docs

* redo intor and getting started sections

* update attestation binding

* more tweaks

* update docs

* Reviewed getting started

Co-authored-by: Jon Jagger <[email protected]>
Co-authored-by: Sami Alajrami <[email protected]>
Co-authored-by: Simon Castagna <[email protected]>
Co-authored-by: Tore Martin Hagen <[email protected]>

* Replace distribution -> deployents

Co-authored-by: Jon Jagger <[email protected]>
Co-authored-by: Sami Alajrami <[email protected]>
Co-authored-by: Simon Castagna <[email protected]>
Co-authored-by: Tore Martin Hagen <[email protected]>

* Fix merge issues with get_familiar_with_Kosli

* Update to getting started: attestations page

* Update attestations doc

* Minor edits

* Explain that template file is optional in getting started

* Better section title

---------

Co-authored-by: JonJagger <[email protected]>
Co-authored-by: Simon Castagna <[email protected]>
Co-authored-by: Sami Alajrami <[email protected]>
Co-authored-by: Tore Martin Hagen <[email protected]>
Co-authored-by: Jon Jagger <[email protected]>
  • Loading branch information
6 people authored Mar 4, 2024
1 parent 3f8245b commit 3a9fff2
Show file tree
Hide file tree
Showing 24 changed files with 464 additions and 875 deletions.
7 changes: 5 additions & 2 deletions .github/workflows/publish_branch_docs.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
# This workflow deploys the docs from any branch to https://staging-docs--kosli-docs.netlify.app/
# It has to be triggered manually in github actions from the branch that must be deployed

name: publish branch docs

on:
Expand Down Expand Up @@ -36,5 +39,5 @@ jobs:
# docs.kosli.com/assets/metadata.json are not present, but we use CLEAR_GLOBS_FILE to preserve
# their versions pushed to docs-main during last release - hence CLEAR_GLOBS_FILE in
# publish_docs.yml, to prevent removing them before copying
# CLEAR_GLOBS_FILE: ".clear-files"
# GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
CLEAR_GLOBS_FILE: ".clear-files"
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
7 changes: 4 additions & 3 deletions charts/k8s-reporter/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ title: Kubernetes Reporter Helm Chart

# k8s-reporter

![Version: 1.3.1](https://img.shields.io/badge/Version-1.3.1-informational?style=flat-square)
![Version: 1.4.0](https://img.shields.io/badge/Version-1.4.0-informational?style=flat-square)

A Helm chart for installing the Kosli K8S reporter as a cronjob.
The chart allows you to create a Kubernetes cronjob and all its necessary RBAC to report running images to Kosli at a given cron schedule.
Expand All @@ -13,6 +13,7 @@ The chart allows you to create a Kubernetes cronjob and all its necessary RBAC t

- A Kubernetes cluster
- Helm v3.0+
- Create a secret for the Kosli API token which will be used for reporting. You can create a secret by running: `kubectl create secret generic <secret-name> --from-literal=<secret-key>=<your-api-key>`

## Installing the chart

Expand Down Expand Up @@ -57,7 +58,7 @@ helm upgrade [RELEASE-NAME] kosli/k8s-reporter
| fullnameOverride | string | `""` | overrides the fullname used for the created k8s resources. It has higher precedence than `nameOverride` |
| image.pullPolicy | string | `"IfNotPresent"` | the kosli reporter image pull policy |
| image.repository | string | `"ghcr.io/kosli-dev/cli"` | the kosli reporter image repository |
| image.tag | string | `"v2.0.0"` | the kosli reporter image tag, overrides the image tag whose default is the chart appVersion. |
| image.tag | string | `"v2.7.2"` | the kosli reporter image tag, overrides the image tag whose default is the chart appVersion. |
| kosliApiToken.secretKey | string | `""` | the name of the key in the secret data which contains the kosli API token |
| kosliApiToken.secretName | string | `""` | the name of the secret containing the kosli API token |
| nameOverride | string | `""` | overrides the name used for the created k8s resources. If `fullnameOverride` is provided, it has higher precedence than this one |
Expand All @@ -74,5 +75,5 @@ helm upgrade [RELEASE-NAME] kosli/k8s-reporter
| serviceAccount.name | string | `""` | the name of the service account to use. If not set and create is true, a name is generated using the fullname template |

----------------------------------------------
Autogenerated from chart metadata using [helm-docs v1.11.0](https://github.com/norwoodj/helm-docs/releases/v1.11.0)
Autogenerated from chart metadata using [helm-docs v1.11.3](https://github.com/norwoodj/helm-docs/releases/v1.11.3)

2 changes: 1 addition & 1 deletion docs.kosli.com/config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,4 +23,4 @@ Params:
caches:
getjson:
dir: :cacheDir/:project
maxAge: 30s
maxAge: 30s
6 changes: 3 additions & 3 deletions docs.kosli.com/content/_index.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Welcome to Kosli Docs
seo_title: Welcome to Kosli Docs
description: Don’t spend hours searching for broken commits and manual changes. Get to the bottom of incidents faster with Kosli.
description: Record all of the changes in your software and business processes so you can prove compliance and maintain security without slowing down.
hideToC: true

hero:
Expand All @@ -12,15 +12,15 @@ hero:
alt_text: Kosli artie reading a book

paragraph: >
Don’t spend hours searching for broken commits and manual changes. Get to the bottom of incidents faster with Kosli. Track and query every change from commit through to production. See the artifacts your CI pipelines are producing and how your environments are changing from the command line or browser.
Record all of the changes in your software and business processes so you can prove compliance and maintain security without slowing down. Track and query every change from the command line or browser.
sections:
title: Dive right in…
blocks:
- title: What is Kosli
image: /images/home/home-concepts.svg
alt_text: Introducing Kosli icon
description: Read about what Kosli consists of and what it offers
description: Understand what Kosli is and how it works
link_text: View >
url: /understand_kosli/what_is_kosli/
- title: Kosli environments
Expand Down
42 changes: 5 additions & 37 deletions docs.kosli.com/content/getting_started/approvals.md
Original file line number Diff line number Diff line change
@@ -1,44 +1,12 @@
---
title: "Part 8: Approvals"
title: "Part 9: Approvals"
bookCollapseSection: false
weight: 280
weight: 300
---
# Part 8: Approvals
# Part 9: Approvals

Whenever an artifact is ready to be deployed to a given [environment](/getting_started/environments/), an additional approval may be created in Kosli. An approval can be requested which will require a manually action, or reported automatically. This will be recorded into Kosli so the decision made outside of your CI system won't be lost.
When an artifact is ready to be deployed to a given [environment](/getting_started/environments/), an approval may be reported to Kosli. An approval can be requested which will require a manual action, or reported automatically. This will be recorded in Kosli so the decision made outside your CI system won't be lost.

When an approval is created for an artifact to a specific environment with the `--environment` flag, Kosli will generate a list of commits to be approved. By default, this list will contain all commits between `HEAD` and the commit of the most recent artifact coming from the same [flow](/getting_started/flows/) found in the given environment. The list can also be specified by providing values for `--newest-commit` and `--oldest-commit`. If you are providing these commits yourself, keep in mind that `--oldest-commit` has to be an ancestor of `--newest-commit`.


## Example

{{< tabs "approvals" "col-no-wrap" >}}

{{< tab "v2" >}}
```
$ kosli report approval project-a-app.bin \
--artifact-type file \
--environment production \
--flow project-a
approval created for artifact: 53c97572093cc107c0caa2906d460ccd65083a4c626f68689e57aafa34b14cbf
```

See [kosli report approval](/client_reference/kosli_report_approval/) and [kosli request approval](/client_reference/kosli_request_approval/) for more details.
{{< /tab >}}

{{< tab "v0.1.x" >}}
```
$ kosli pipeline approval report project-a-app.bin \
--artifact-type file \
--newest-commit $(git rev-parse HEAD) \
--oldest-commit $(git rev-parse HEAD~1) \
--pipeline project-a
approval created for artifact: 53c97572093cc107c0caa2906d460ccd65083a4c626f68689e57aafa34b14cbf
```

See [kosli pipeline approval report](/legacy_ref/v0.1.41/kosli_pipeline_approval_report/) and [kosli pipeline approval request](/legacy_ref/v0.1.41/kosli_pipeline_approval_request/) for more details.
{{< /tab >}}

{{< /tabs >}}
See [request approval](/client_reference/kosli_request_approval/) and [report approval](/client_reference/kosli_report_approval/) for usage details and examples.
54 changes: 19 additions & 35 deletions docs.kosli.com/content/getting_started/artifacts.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,55 +5,39 @@ weight: 260
---
# Part 6: Artifacts

## Report artifacts
In software processes, you typically generate one or more artifacts that are deployed or distributed, such as docker images, archives, binaries, etc. You can ensure traceability for the creation of these artifacts by attesting them to Kosli, thereby establishing a binary provenance for each one.

To report an artifact to Kosli, you need its SHA256 fingerprint. You can either provide the fingerprint yourself, or let Kosli CLI calculate it for you - we'll need the artifact available while running the reporting command to do that.
You also need to provide the name of the Kosli flow you want to report the artifact to.
## Binary provenance

You should also have long enough git history in your local git repo clone to let Kosli calculate the artifact's changelog (the list of commits from the new artifact back to the previous artifact reported to the same Kosli flow).
If you use shallow clone in your CI, Kosli won't be able to generate the changelog but the artifact reporting will **NOT** fail. Kosli collects the changelog commits on best-effort basis.
Binary provenance for artifacts refers to the ability to trace and verify the origins, history, and journey of the artifacts throughout their lifecycle. This involves recording immutable attestations about the artifact creation, risk controls performed on it, deployments, and execution/usage.

The fingerprint (SHA256 checksum of the file/directory/docker image) of the artifact will be stored in Kosli. The fingerprint can't be changed, it becomes a unique identifier of the artifact in Kosli, used - among other things - to connect it with the data reported from your runtime environments.
Artifacts are uniquely identified by their SHA256 fingerprints. When attesting an artifact to Kosli, you have the option to either provide the fingerprint manually or allow Kosli CLI to calculate it automatically for you.

Fingerprints of all the **running** artifacts, recorded with the Kosli CLI are also stored and **compared with** fingerprints of the artifacts you have built and **reported** to Kosli so you always know if you're running things you have built or if you have no provenance for them.
By leveraging the artifact's fingerprint, Kosli can establish connections between the creation of the artifact and its runtime-related events, such as when the artifact starts or ceases execution within a specific environment.

Some of the required flags will be automatically resolved if you're using one of the [supported CI systems](/integrations/ci_cd/).
By establishing and maintaining binary provenance for artifacts, Kosli enables you to:

### Example
1. **Track Changes**: Trace how your Flow artifacts change over time.
2. **Identify Sources**: Understand where your artifacts originated from, which can help in identifying vulnerabilities or issues.
3. **Monitor Compliance**: Ensure that the artifacts adhere to your compliance requirements.
4. **Enable Audits**: Access audit packages on demand allowing audits and investigations into the software supply chain.
5. **Enhance Trust**: Build trust among users, customers, and stakeholders by providing transparent and verified information about the software's history.

{{< tabs "commands" "col-no-wrap" >}}
## Attesting artifacts

{{< tab "v2" >}}
```
$ kosli report artifact project-a-app.bin \
--artifact-type file \
--build-url https://exampleci.com \
--commit-url https://github.com/ProjectA/ProjectAApp/commit/e67f2f2b121f9325ebf166b7b3c707f73cb48b14 \
--git-commit e67f2f2b121f9325ebf166b7b3c707f73cb48b14 \
--flow project-a
artifact project-a-app.bin was reported with fingerprint: 53c97572093cc107c0caa2906d460ccd65083a4c626f68689e57aafa34b14cbf
```
See [kosli report artifact](/client_reference/kosli_report_artifact/) for more details.

{{< /tab >}}
To attest an artifact, you can run a command similar to the one below:

{{< tab "v0.1.x" >}}
```
$ kosli pipeline artifact report creation project-a-app.bin \
```shell
$ kosli attest artifact project-a-app.bin \
--artifact-type file \
--build-url https://exampleci.com \
--commit-url https://github.com/ProjectA/ProjectAApp/commit/e67f2f2b121f9325ebf166b7b3c707f73cb48b14 \
--git-commit e67f2f2b121f9325ebf166b7b3c707f73cb48b14 \
--pipeline project-a
artifact project-a-app.bin was reported with fingerprint: 53c97572093cc107c0caa2906d460ccd65083a4c626f68689e57aafa34b14cbf
--flow project-a \
--trail trail-1 \
--name backend
```
See [kosli pipeline artifact report creation](/legacy_ref/v0.1.41/kosli_pipeline_artifact_report_creation/) for more details.

{{< /tab >}}

{{< /tabs >}}
See [kosli attest artifact](/client_reference/kosli_attest_artifact/) for more details.



160 changes: 160 additions & 0 deletions docs.kosli.com/content/getting_started/attestations.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@
---
title: "Part 7: Attestations"
bookCollapseSection: false
weight: 270
---
# Part 7: Attestations

Attestations are how you record the facts your care about in your software supply chain. They are the evidence that you have performed certain activities, such as running tests, security scans, or ensuring that a certain requirement is met.

Kosli allows you to report different types of attestations about artifacts and trails. For some types, Kosli will process the evidence you provide and conclude whether the evidence proves compliance or otherwise.

Let's take a look at how to make attestations to Kosli.

The following template is expecting 4 attestations, one for each `name`.

```yml
version: 1
trail:
attestations:
- name: jira-ticket
type: jira
artifacts:
- name: backend
attestations:
- name: unit-tests
type: junit
- name: security-scan
type: snyk
```
It expects `jira-ticket` on the trail, the `backend` artifact, with `unit-tests` and `security-scan` attached to it. When you make an attestation, you have the choice of what `name` to attach it to:

## Make the `jira-ticket` attestation to a trail

The `jira-ticket` attestation belongs to a single trail and is not linked to a specific artifact. In this example, the id of the trail is the git commit.

```shell
$ kosli attest jira \
--flow backend-ci \
--trail $(git rev-parse HEAD) \
--name jira-ticket
...
```

## Make the `unit-test` attestation to the `backend` artifact

Some attestations are attached to a specific artifact, like the unit tests for the `backend` artifact. Often, evidence like unit tests are created before the artifact is built. To attach the evidence to the artifact before its creation, use `backend` (the artifact's `name` from the template), as well as `unit-tests` (the attestation's `name` from the template).

```shell
$ kosli attest junit \
--name backend.unit-tests \
--flow backend-ci \
--trail $(git rev-parse HEAD) \
...
```

This attestation belongs to any artifact attested with the matching `name` from the template (in this example `backend`) and a matching git commit.

## Make the `backend` artifact attestation

Once the artifact has been built, it can be attested with the following command.

```shell
$ kosli attest artifact my_company/backend:latest \
--artifact-type docker \
--flow backend-ci \
--trail $(git rev-parse HEAD) \
--name backend
...
```

The Kosli CLI will calculate the fingerprint of the docker image called `my_company/backend:latest` and attest it as the `backend` artifact `name` in the trail.

{{< hint info >}}
### Automatically gather git commit and CI environment information
In all attestation commands the Kosli CLI will automatically gather the git commit and other information from the current git repository and the [CI environment](https://docs.kosli.com/integrations/ci_cd/). This is how the git commit is used to match attestation to artifacts.
{{< /hint >}}

## Make the `security-scan` attestation to the `backend` artifact

Often, evidence like snyk reports are created after the artifact is built. In this case, you can attach the evidence to the artifact after its creation. Use `backend` (the artifact's `name` from the template), as well as `security-scan` (the attestation's `name` from the template) to name the attestation.

The following attestation will only belong to the artifact `my_company/backend:latest` attested above and its fingerprint, in this case calculated by the Kosli CLI.

```shell
$ kosli attest snyk \
--artifact-type docker my_company/backend:latest \
--name backend.security-scan \
--flow backend-ci \
--trail $(git rev-parse HEAD)
...
```

{{< hint info >}}
### Attestation immutability

Attestations are append-only immutable records. You can report the same attestation multiple times, and each report will be recorded. However, only the latest version of the attestation is considered when evaluating trail or artifact compliance.
{{< /hint >}}

## Evidence Vault

Along with attestations data, you can attach additional supporting evidence files. These will be securely stored in Kosli's **Evidence Vault** and can easily be retrieved when needed. Alternatively, you can store the evidence files in your own preferred storage and only attach links to it in the Kosli attestation.

{{< hint info >}}

For `JUnit` attestations (see below), Kosli automatically stores the JUnit XML results files in the Evidence Vault. You can disable this by setting `--upload-results=false`

{{< /hint >}}

## Attestation types

Currently, we support the following types of evidence:

### Pull requests

If you use GitHub, Bitbucket, Gitlab or Azure DevOps you can use Kosli to verify if a given git commit comes from a pull/merge request.

{{< hint warning >}}
Currently, the status of the PR does NOT impact the compliance status of the attestation.
{{< /hint >}}

If there is no pull request for the commit, the attestation will be reported as `non-compliant`. You can choose to short-circuit execution in case pull request is missing by using the `--assert` flag.

See the CLI reference for the following commands for more details and examples:

- [attest Github PR ](/client_reference/kosli_attest_pullrequest_github/)
- [attest Bitbucket PR ](/client_reference/kosli_attest_pullrequest_bitbucket/)
- [attest Gitlab PR ](/client_reference/kosli_attest_pullrequest_gitlab/)
- [attest Azure Devops PR ](/client_reference/kosli_attest_pullrequest_azure/)


### JUnit test results

If you produce your test results in JUnit format, you can attest the test results to Kosli. Kosli will analyze the JUnit results and determine the compliance status based on whether any tests have failed and/or errored or not.

See [attest JUnit results to an artifact or a trail](/client_reference/kosli_attest_junit/) for usage details and examples.

### Snyk security scans

You can report results of a Snyk security scan to Kosli and it will analyze the Snyk scan results and determine the compliance status based on whether vulnerabilities where found or not.

See [attest Snyk results to an artifact or a trail](/client_reference/kosli_attest_snyk/) for usage details and examples.

### Jira issues

You can use the Jira attestation to verify that a git commit or branch contains a reference to a Jira issue and that an issue with the same reference does exist in Jira.

If Jira reference is found in a commit message, that reference will be reported as evidence. If the reference is not found in the commit message, Kosli CLI will check if it's a part of a branch name.

Kosli CLI will also verify and report if the detected issue reference is found and accessible on Jira (reported as compliant) or not (reported as non compliant).

See [attest Jira issue to an artifact or a trail](/client_reference/kosli_attest_jira/) for usage details and examples.

### Generic

If Kosli doesn't support the type of the attestation you'd like to attach, you can use the generic type.

Use `--compliant=false` if you want to report a given evidence as non-compliant.

See [report generic attestation to an artifact or a trail](/client_reference/kosli_attest_generic/) for usage details and examples.
Loading

0 comments on commit 3a9fff2

Please sign in to comment.