Release: start adding oci image support to fingerprint and attest art…

…ifact commands (#381)

* Started on kosli fingerprint oci command

* fix dependencies

* explicitly set the docker sdk api version to use in tests

* remove unneeded comment and debug print

* remove redundant step in dockerfile

---------

Co-authored-by: Tore Martin Hagen <[email protected]>

docs.kosli.com/assets/metadata.json

@@ -1 +1 @@
-{"currentversion": "v2.11.0"}
+{"currentversion": "v2.11.1"}

docs.kosli.com/content/legacy_ref/v2.10.15/_index.md

@@ -1,7 +1,7 @@
 ---
 title: v2.10.15
 bookCollapseSection: true
-weight: 603
+weight: 604
 ---
 
 # v2.10.15

docs.kosli.com/content/legacy_ref/v2.10.16/_index.md

@@ -1,7 +1,7 @@
 ---
 title: v2.10.16
 bookCollapseSection: true
-weight: 602
+weight: 603
 ---
 
 # v2.10.16

docs.kosli.com/content/legacy_ref/v2.10.17/_index.md

@@ -1,7 +1,7 @@
 ---
 title: v2.10.17
 bookCollapseSection: true
-weight: 601
+weight: 602
 ---
 
 # v2.10.17

docs.kosli.com/content/legacy_ref/v2.10.18/_index.md

@@ -1,7 +1,7 @@
 ---
 title: v2.10.18
 bookCollapseSection: true
-weight: 600
+weight: 601
 ---
 
 # v2.10.18

docs.kosli.com/content/legacy_ref/v2.11.0/_index.md

@@ -0,0 +1,7 @@
+---
+title: v2.11.0
+bookCollapseSection: true
+weight: 600
+---
+
+# v2.11.0

docs.kosli.com/content/legacy_ref/v2.10.14/kosli_attest_artifact.md → docs.kosli.com/content/legacy_ref/v2.11.0/kosli_attest_artifact.md

@@ -9,7 +9,9 @@ deprecated: false
 ## Synopsis
 
 Attest an artifact creation to a Kosli flow.  
-The artifact SHA256 fingerprint is calculated (based on the `--artifact-type` flag and the artifact name/path argument) or can be provided directly (with the `--fingerprint` flag).
+The artifact SHA256 fingerprint is calculated (based on the `--artifact-type` flag and the artifact name/path argument) or can be provided directly (with the `--fingerprint` flag).To specify paths in a directory artifact that should always be excluded from the SHA256 calculation, you can add a `.kosli_ignore` file to the root of the artifact.
+Each line should specify a relative path or path glob to be ignored. You can include comments in this file, using `#`.
+The `.kosli_ignore` will be treated as part of the artifact like any other file,unless it is explicitly ignored itself.
 This command requires access to a git repo to associate the artifact to the git commit it is originating from. 
 You can optionally redact some of the git commit data sent to Kosli using `--redact-commit-info`
 

docs.kosli.com/content/legacy_ref/v2.10.14/kosli_attest_junit.md → docs.kosli.com/content/legacy_ref/v2.11.0/kosli_attest_junit.md

@@ -8,7 +8,9 @@ deprecated: false
 
 ## Synopsis
 
-Report a junit attestation to an artifact or a trail in a Kosli flow.  
+Report a junit attestation to an artifact or a trail in a Kosli flow.
+JUnit xml files are read from the `--results-dir` directory which defaults to the current directory.
+The xml files are automatically uploaded as `--attachments` via the `--upload-results` flag which defaults to `true`.  
 
 The attestation can be bound to a trail using the trail name.
 

docs.kosli.com/content/legacy_ref/v2.10.14/kosli_attest_sonar.md → docs.kosli.com/content/legacy_ref/v2.11.0/kosli_attest_sonar.md

@@ -13,13 +13,16 @@ Retrieves results for the specified scan from SonarCloud or SonarQube and attest
 The results are parsed to find the status of the project's quality gate which is used to determine the attestation's compliance status.
 
 The scan to be retrieved can be specified in two ways:
-1. Using metadata created by the Sonar scanner. By default this is located within a temporary .scannerwork folder in the repo base directory.
+1. (Default) Using metadata created by the Sonar scanner. By default this is located within a temporary .scannerwork folder in the repo base directory.
 If you have overriden the location of this folder by passing parameters to the Sonar scanner, or are running Kosli's CLI locally outside the repo's base directory,
 you can provide the correct path using the --sonar-working-dir flag. This metadata is generated by a specific scan, allowing Kosli to retrieve the results of that scan.
 2. Providing the Sonar project key and the revision of the scan (plus the SonarQube server URL if relevant). If running the Kosli CLI in some CI/CD pipeline, the revision
 is defaulted to the commit SHA. If you are running the command locally, or have overriden the revision in SonarCloud/SonarQube via parameters to the Sonar scanner, you can
 provide the correct revision using the --sonar-revision flag. Kosli then finds the scan results for the specified project key and revision.
 
+Note that if your project is very large and you are using SonarCloud's automatic analysis, it is possible for the attest sonar command to run before the SonarCloud scan is completed.
+In this case, we recommend using Kosli's Sonar webhook integration ( https://docs.kosli.com/integrations/sonar/ ) rather than the CLI to attest the scan results.
+
 
 The attestation can be bound to a trail using the trail name.
 

docs.kosli.com/content/legacy_ref/v2.10.14/kosli_begin_trail.md → docs.kosli.com/content/legacy_ref/v2.11.0/kosli_begin_trail.md

@@ -11,7 +11,10 @@ deprecated: false
 Begin or update a Kosli flow trail.
 
 You can optionally associate the trail to a git commit using `--commit` (requires access to a git repo). And you  
-can optionally redact some of the git commit data sent to Kosli using `--redact-commit-info`
+can optionally redact some of the git commit data sent to Kosli using `--redact-commit-info`.
+
+`TRAIL-NAME`s must start with a letter or number, and only contain letters, numbers, `.`, `-`, `_`, and `~`.
+
 
 ```shell
 kosli begin trail TRAIL-NAME [flags]
@@ -30,7 +33,7 @@ kosli begin trail TRAIL-NAME [flags]
 |    -o, --origin-url string  |  [optional] The url pointing to where the attestation came from or is related. (defaulted to the CI url in some CIs: https://docs.kosli.com/ci-defaults ).  |
 |        --redact-commit-info strings  |  [optional] The list of commit info to be redacted before sending to Kosli. Allowed values are one or more of [author, message, branch].  |
 |        --repo-root string  |  [defaulted] The directory where the source git repository is available. Only used if --commit is used. (default ".")  |
-|    -f, --template-file string  |  [optional] The path to a yaml template file. Cannot be used together with --use-empty-template  |
+|    -f, --template-file string  |  [optional] The path to a yaml template file.  |
 |    -u, --user-data string  |  [optional] The path to a JSON file containing additional data you would like to attach to the flow trail.  |
 
 

docs.kosli.com/content/legacy_ref/v2.10.14/kosli_create_environment.md → docs.kosli.com/content/legacy_ref/v2.11.0/kosli_create_environment.md

@@ -35,6 +35,8 @@ Logical environments are used for grouping of physical environments. For instanc
 be grouped into logical environment **prod**. Logical environments are view-only, you can not report snapshots
 to them.
 
+`ENVIRONMENT-NAME`s must start with a letter or number, and only contain letters, numbers, `.`, `-`, `_`, and `~`.
+
 
 ```shell
 kosli create environment ENVIRONMENT-NAME [flags]

docs.kosli.com/content/legacy_ref/v2.10.14/kosli_create_flow.md → docs.kosli.com/content/legacy_ref/v2.11.0/kosli_create_flow.md

@@ -11,6 +11,9 @@ deprecated: false
 Create or update a Kosli flow.
 You can specify flow parameters in flags.
 
+`FLOW-NAME`s must start with a letter or number, and only contain letters, numbers, `.`, `-`, `_`, and `~`.
+
+
 ```shell
 kosli create flow FLOW-NAME [flags]
 ```

docs.kosli.com/content/legacy_ref/v2.10.14/kosli_fingerprint.md → docs.kosli.com/content/legacy_ref/v2.11.0/kosli_fingerprint.md

@@ -22,6 +22,10 @@ glob patterns.
 The supported glob pattern syntax is what is documented here: https://pkg.go.dev/path/filepath#Match , 
 plus the ability to use recursive globs "**"
 
+To specify paths in a directory artifact that should always be excluded from the SHA256 calculation, you can add a `.kosli_ignore` file to the root of the artifact.
+Each line should specify a relative path or path glob to be ignored. You can include comments in this file, using `#`.
+The `.kosli_ignore` will be treated as part of the artifact like any other file,unless it is explicitly ignored itself.
+
 ```shell
 kosli fingerprint {IMAGE-NAME | FILE-PATH | DIR-PATH} [flags]
 ```

docs.kosli.com/content/legacy_ref/v2.11.0/kosli_join_environment.md

@@ -0,0 +1,49 @@
+---
+title: "kosli join environment"
+beta: false
+deprecated: false
+---
+
+# kosli join environment
+
+## Synopsis
+
+Join a physical environment to a logical environment.
+
+```shell
+kosli join environment [flags]
+```
+
+## Flags
+| Flag | Description |
+| :--- | :--- |
+|    -D, --dry-run  |  [optional] Run in dry-run mode. When enabled, no data is sent to Kosli and the CLI exits with 0 exit code regardless of any errors.  |
+|    -h, --help  |  help for environment  |
+|        --logical string  |  [required] The logical environment.  |
+|        --physical string  |  [required] The physical environment.  |
+
+
+## Flags inherited from parent commands
+| Flag | Description |
+| :--- | :--- |
+|    -a, --api-token string  |  The Kosli API token.  |
+|    -c, --config-file string  |  [optional] The Kosli config file path. (default "kosli")  |
+|        --debug  |  [optional] Print debug logs to stdout. A boolean flag https://docs.kosli.com/faq/#boolean-flags (default false)  |
+|    -H, --host string  |  [defaulted] The Kosli endpoint. (default "https://app.kosli.com")  |
+|        --http-proxy string  |  [optional] The HTTP proxy URL including protocol and port number. e.g. 'http://proxy-server-ip:proxy-port'  |
+|    -r, --max-api-retries int  |  [defaulted] How many times should API calls be retried when the API host is not reachable. (default 3)  |
+|        --org string  |  The Kosli organization.  |
+
+
+## Examples Use Cases
+
+**join a physical environment to a logical environment**
+
+```shell
+kosli join environment \
+	--physical prod-k8 \
+	--logical prod \
+	--api-token yourAPIToken \
+	--org yourOrgName
+```
+

docs.kosli.com/content/legacy_ref/v2.10.14/kosli_snapshot_ecs.md → docs.kosli.com/content/legacy_ref/v2.11.0/kosli_snapshot_ecs.md

@@ -8,7 +8,8 @@ deprecated: false
 
 ## Synopsis
 
-Report a snapshot of running containers in an AWS ECS cluster or service to Kosli.  
+Report a snapshot of running containers in one or more AWS ECS cluster(s) to Kosli.  
+Skip `--clusters` and `--clusters-regex` to report all clusters in a given AWS account. Or use `--exclude` and/or `--exclude-regex` to report all clusters excluding some.
 The reported data includes container image digests and creation timestamps.
 
 To authenticate to AWS, you can either:  
@@ -30,10 +31,12 @@ kosli snapshot ecs ENVIRONMENT-NAME [flags]
 |        --aws-key-id string  |  The AWS access key ID.  |
 |        --aws-region string  |  The AWS region.  |
 |        --aws-secret-key string  |  The AWS secret access key.  |
-|    -C, --cluster string  |  The name of the ECS cluster.  |
+|        --clusters strings  |  [optional] The comma-separated list of ECS cluster names to snapshot. Can't be used together with --exclude or --exclude-regex.  |
+|        --clusters-regex strings  |  [optional] The comma-separated list of ECS cluster name regex patterns to snapshot. Can't be used together with --exclude or --exclude-regex.  |
 |    -D, --dry-run  |  [optional] Run in dry-run mode. When enabled, no data is sent to Kosli and the CLI exits with 0 exit code regardless of any errors.  |
+|        --exclude strings  |  [optional] The comma-separated list of ECS cluster names to exclude. Can't be used together with --exclude or --exclude-regex.  |
+|        --exclude-regex strings  |  [optional] The comma-separated list of ECS cluster name regex patterns to exclude. Can't be used together with --clusters or --clusters-regex.  |
 |    -h, --help  |  help for ecs  |
-|    -s, --service-name string  |  [optional] The name of the ECS service.  |
 
 
 ## Flags inherited from parent commands
@@ -58,7 +61,7 @@ export AWS_ACCESS_KEY_ID=yourAWSAccessKeyID
 export AWS_SECRET_ACCESS_KEY=yourAWSSecretAccessKey
 
 kosli snapshot ecs yourEnvironmentName \
-	--cluster yourECSClusterName \
+	--clusters yourECSClusterName \
 	--api-token yourAPIToken \
 	--org yourOrgName
 
@@ -72,22 +75,33 @@ export AWS_ACCESS_KEY_ID=yourAWSAccessKeyID
 export AWS_SECRET_ACCESS_KEY=yourAWSSecretAccessKey
 
 kosli snapshot ecs yourEnvironmentName \
-	--cluster yourECSClusterName \
+	--clusters yourECSClusterName \
 	--service-name yourECSServiceName \
 	--api-token yourAPIToken \
 	--org yourOrgName
 
 ```
 
-**report what is running in in a specific AWS ECS service within a cluster (AWS auth provided in flags)**
+**report what is running in all ECS clusters in an AWS account (AWS auth provided in flags)**
+
+```shell
+kosli snapshot ecs yourEnvironmentName \
+	--aws-key-id yourAWSAccessKeyID \
+	--aws-secret-key yourAWSSecretAccessKey \
+	--aws-region yourAWSRegion \
+	--api-token yourAPIToken \
+	--org yourOrgName
+
+```
+
+**report what is running in all ECS clusters in an AWS account except for clusters with names matching given regex patterns**
 
 ```shell
 kosli snapshot ecs yourEnvironmentName \
-	--cluster yourECSClusterName \
-	--service-name yourECSServiceName \
 	--aws-key-id yourAWSAccessKeyID \
 	--aws-secret-key yourAWSSecretAccessKey \
 	--aws-region yourAWSRegion \
+	--exclude-regex "those-names.*" \
 	--api-token yourAPIToken \
 	--org yourOrgName
 ```

docs.kosli.com/content/legacy_ref/v2.10.14/kosli_snapshot_k8s.md → docs.kosli.com/content/legacy_ref/v2.11.0/kosli_snapshot_k8s.md

@@ -9,6 +9,7 @@ deprecated: false
 ## Synopsis
 
 Report a snapshot of running pods in a K8S cluster or namespace(s) to Kosli.  
+Skip `--namespaces` and `--namespaces-regex` to report all pods in all namespaces in a cluster.
 The reported data includes pod container images digests and creation timestamps. You can customize the scope of reporting
 to include or exclude namespaces.
 
@@ -20,10 +21,12 @@ kosli snapshot k8s ENVIRONMENT-NAME [flags]
 | Flag | Description |
 | :--- | :--- |
 |    -D, --dry-run  |  [optional] Run in dry-run mode. When enabled, no data is sent to Kosli and the CLI exits with 0 exit code regardless of any errors.  |
-|    -x, --exclude-namespaces strings  |  [conditional] The comma separated list of namespaces regex patterns NOT to report artifacts info from. Can't be used together with --namespace.  |
+|    -x, --exclude-namespaces strings  |  [optional] The comma separated list of namespaces names to exclude from reporting artifacts info from. Can't be used together with --namespaces or --namespaces-regex.  |
+|        --exclude-namespaces-regex strings  |  [optional] The comma separated list of namespaces regex patterns to exclude from reporting artifacts info from. Can't be used together with --namespaces or --namespaces-regex.  |
 |    -h, --help  |  help for k8s  |
 |    -k, --kubeconfig string  |  [defaulted] The kubeconfig path for the target cluster. (default "$HOME/.kube/config")  |
-|    -n, --namespaces strings  |  [conditional] The comma separated list of namespaces regex patterns to report artifacts info from. Can't be used together with --exclude-namespace.  |
+|    -n, --namespaces strings  |  [optional] The comma separated list of namespaces names to report artifacts info from. Can't be used together with --exclude-namespaces or --exclude-namespaces-regex.  |
+|        --namespaces-regex strings  |  [optional] The comma separated list of namespaces regex patterns to report artifacts info from. Can't be used together with --exclude-namespaces --exclude-namespaces-regex.  |
 
 
 ## Flags inherited from parent commands

docs.kosli.com/content/legacy_ref/v2.10.14/kosli_snapshot_lambda.md → docs.kosli.com/content/legacy_ref/v2.11.0/kosli_snapshot_lambda.md

@@ -9,7 +9,7 @@ deprecated: false
 ## Synopsis
 
 Report a snapshot of artifacts deployed as one or more AWS Lambda functions and their digests to Kosli.  
-Skip `--function-names` to report all functions in a given AWS account.
+Skip `--function-names` and `--function-names-regex` to report all functions in a given AWS account. Or use `--exclude` and/or `--exclude-regex` to report all functions excluding some.
 
 To authenticate to AWS, you can either:  
   1) provide the AWS static credentials via flags or by exporting the equivalent KOSLI env vars (e.g. KOSLI_AWS_KEY_ID)  
@@ -31,7 +31,10 @@ kosli snapshot lambda ENVIRONMENT-NAME [flags]
 |        --aws-region string  |  The AWS region.  |
 |        --aws-secret-key string  |  The AWS secret access key.  |
 |    -D, --dry-run  |  [optional] Run in dry-run mode. When enabled, no data is sent to Kosli and the CLI exits with 0 exit code regardless of any errors.  |
-|        --function-names strings  |  [optional] The comma-separated list of AWS Lambda function names to be reported.  |
+|        --exclude strings  |  [optional] The comma-separated list of AWS Lambda function names to be excluded. Cannot be used together with --function-names  |
+|        --exclude-regex strings  |  [optional] The comma-separated list of name regex patterns for AWS Lambda functions to be excluded. Cannot be used together with --function-names. Allowed regex patterns are described in https://github.com/google/re2/wiki/Syntax  |
+|        --function-names strings  |  [optional] The comma-separated list of AWS Lambda function names to be reported. Cannot be used together with --exclude or --exclude-regex.  |
+|        --function-names-regex strings  |  [optional] The comma-separated list of AWS Lambda function names regex patterns to be reported. Cannot be used together with --exclude or --exclude-regex.  |
 |    -h, --help  |  help for lambda  |
 
 
@@ -62,6 +65,21 @@ kosli snapshot lambda yourEnvironmentName \
 
 ```
 
+**report all (excluding some) Lambda functions running in an AWS account (AWS auth provided in env variables)**
+
+```shell
+export AWS_REGION=yourAWSRegion
+export AWS_ACCESS_KEY_ID=yourAWSAccessKeyID
+export AWS_SECRET_ACCESS_KEY=yourAWSSecretAccessKey
+
+kosli snapshot lambda yourEnvironmentName \
+    --exclude function1,function2 \
+	--exclude-regex "^not-wanted.*" \
+	--api-token yourAPIToken \
+	--org yourOrgName
+
+```
+
 **report what is running in the latest version of an AWS Lambda function (AWS auth provided in env variables)**
 
 ```shell
@@ -76,6 +94,20 @@ kosli snapshot lambda yourEnvironmentName \
 
 ```
 
+**report what is running in the latest version of AWS Lambda functions that match a name regex**
+
+```shell
+export AWS_REGION=yourAWSRegion
+export AWS_ACCESS_KEY_ID=yourAWSAccessKeyID
+export AWS_SECRET_ACCESS_KEY=yourAWSSecretAccessKey
+
+kosli snapshot lambda yourEnvironmentName \
+	--function-names-regex yourFunctionNameRegexPattern \
+	--api-token yourAPIToken \
+	--org yourOrgName
+
+```
+
 **report what is running in the latest version of multiple AWS Lambda functions (AWS auth provided in env variables)**
 
 ```shell

docs.kosli.com/content/legacy_ref/v2.10.14/kosli_snapshot_path.md → docs.kosli.com/content/legacy_ref/v2.11.0/kosli_snapshot_path.md

@@ -14,7 +14,9 @@ You can exclude certain paths or patterns from the artifact fingerprint using `-
 The supported glob pattern syntax is what is documented here: https://pkg.go.dev/path/filepath#Match , 
 plus the ability to use recursive globs "**"
 
-
+To specify paths in a directory artifact that should always be excluded from the SHA256 calculation, you can add a `.kosli_ignore` file to the root of the artifact.
+Each line should specify a relative path or path glob to be ignored. You can include comments in this file, using `#`.
+The `.kosli_ignore` will be treated as part of the artifact like any other file,unless it is explicitly ignored itself.
 
 ```shell
 kosli snapshot path ENVIRONMENT-NAME [flags]

docs.kosli.com/content/legacy_ref/v2.10.14/kosli_snapshot_paths.md → docs.kosli.com/content/legacy_ref/v2.11.0/kosli_snapshot_paths.md

@@ -20,6 +20,10 @@ glob patterns.
 The supported glob pattern syntax is what is documented here: https://pkg.go.dev/path/filepath#Match , 
 plus the ability to use recursive globs "**"
 
+To specify paths in a directory artifact that should always be excluded from the SHA256 calculation, you can add a `.kosli_ignore` file to the root of the artifact.
+Each line should specify a relative path or path glob to be ignored. You can include comments in this file, using `#`.
+The `.kosli_ignore` will be treated as part of the artifact like any other file,unless it is explicitly ignored itself.
+
 This is an example YAML paths spec file:
 ```yaml
 version: 1

docs.kosli.com/content/legacy_ref/v2.10.14/kosli_snapshot_s3.md → docs.kosli.com/content/legacy_ref/v2.11.0/kosli_snapshot_s3.md

@@ -21,6 +21,9 @@ More details can be found here: https://aws.github.io/aws-sdk-go-v2/docs/configu
 You can report the entire bucket content, or filter some of the content using `--include` and `--exclude`.
 In all cases, the content is reported as one artifact. If you wish to report separate files/dirs within the same bucket as separate artifacts, you need to run the command twice.
 
+To specify paths in a directory artifact that should always be excluded from the SHA256 calculation, you can add a `.kosli_ignore` file to the root of the artifact.
+Each line should specify a relative path or path glob to be ignored. You can include comments in this file, using `#`.
+The `.kosli_ignore` will be treated as part of the artifact like any other file,unless it is explicitly ignored itself.
 
 ```shell
 kosli snapshot s3 ENVIRONMENT-NAME [flags]

docs.kosli.com/content/legacy_ref/v2.10.14/kosli_snapshot_server.md → docs.kosli.com/content/legacy_ref/v2.11.0/kosli_snapshot_server.md

@@ -19,6 +19,10 @@ glob patterns.
 The supported glob pattern syntax is what is documented here: https://pkg.go.dev/path/filepath#Match , 
 plus the ability to use recursive globs "**"
 
+To specify paths in a directory artifact that should always be excluded from the SHA256 calculation, you can add a `.kosli_ignore` file to the root of the artifact.
+Each line should specify a relative path or path glob to be ignored. You can include comments in this file, using `#`.
+The `.kosli_ignore` will be treated as part of the artifact like any other file,unless it is explicitly ignored itself.
+
 ```shell
 kosli snapshot server ENVIRONMENT-NAME [flags]
 ```