Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 43 additions & 2 deletions administration/managing_custom_attestation_types/overview.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Managing Custom Attestation Types
description: Learn how to manage Kosli custom attestation types via Terraform, including creating and importing types with JSON Schema and jq evaluation rules.
description: Learn how to manage Kosli custom attestation types via Terraform, including creating and importing types with JSON Schema, jq evaluation rules, and summaries.
---

The preferred way to manage custom attestation types is via the <Tooltip tip="An official HashiCorp-registered Terraform provider that lets you manage Kosli resources (environments, flows, policies, etc.) as infrastructure as code." cta="View on Terraform Registry" href="https://registry.terraform.io/providers/kosli-dev/kosli/latest/docs/">Kosli Terraform provider</Tooltip>, so your Kosli configuration is version-controlled alongside your infrastructure. You can also manage custom attestation types through the Kosli CLI.
Expand All @@ -13,8 +13,12 @@ Custom attestation types define how Kosli validates evidence from tools that don

- A **JSON Schema** (optional) that defines the expected structure of attestation data
- **jq rules** (optional) that evaluate the data to determine compliance
- A **summary** (optional) — ordered, labeled jq expressions that Kosli renders as rows on the attestation detail page

At least one of the two must be provided.
At least one of the schema or the jq rules must be provided. The summary is independent of both:
it only affects how attestations of the type are displayed, never whether they are compliant. See
[Summarizing custom attestations](/getting_started/attestations#summarizing-custom-attestations) for
how summaries render.

## Create a custom attestation type

Expand Down Expand Up @@ -72,6 +76,43 @@ resource "kosli_custom_attestation_type" "deployment_record" {
}
```

### Adding a summary

`summary` takes a JSON array of `{name, expression}` objects. Each expression is a jq expression
evaluated against the attestation data when the attestation is displayed, and the entries render in
the order given. A string value beginning with `http://` or `https://` renders as a clickable link.

```hcl
resource "kosli_custom_attestation_type" "vulnerability_scan" {
name = "vulnerability-scan"
description = "Validates vulnerability scan results"

jq_rules = [".critical_vulnerabilities == 0"]

summary = jsonencode([
{ name = "Critical", expression = ".critical_vulnerabilities" },
{ name = "High", expression = ".high_vulnerabilities" },
{ name = "Scanner", expression = ".scanner_version" },
{ name = "Report", expression = ".report_url" },
])
}
```

Use `file()` instead of `jsonencode()` to keep the summary in a standalone JSON file, so the same
definition can be shared with other tooling:

```hcl
summary = file("${path.module}/summaries/vulnerability-scan.json")
```

<Note>
The summary is part of the versioned type definition, so changing it creates a new version of the
attestation type — exactly like changing the schema or the jq rules. Removing `summary` from a type
that had one clears the summary on the new version, so attestations reported against it fall back to
showing the jq evaluation results as a pass/fail checklist. Attestations reported against an earlier
version keep the summary that version defined.
</Note>
Comment thread
dangrondahl marked this conversation as resolved.

## Import an existing custom attestation type

If you have custom attestation types created via the CLI, you can bring them under Terraform management by importing them into your <Tooltip tip="The Terraform state file tracks the mapping between your configuration and real-world resources. Importing adds an existing resource to this state without recreating it.">Terraform state</Tooltip>.
Expand Down
67 changes: 63 additions & 4 deletions getting_started/attestations.md
Original file line number Diff line number Diff line change
Expand Up @@ -329,15 +329,15 @@ Currently, we support the following types of evidence:
You could create a custom attestation type called `coverage-metrics` using a [jq expression](https://jqlang.org/manual/) rule defining a minimum line coverage of 95%:

```bash
kosli create attestation-type coverage-metrics
kosli create attestation-type coverage-metrics \
--jq=".code.lines.missed / .code.lines.total * 100 <= 5"
```

You could then make your custom attestation with the json file:
```bash
kosli attest custom
--type=coverage-metrics
--attestation-data=unit-test-coverage.json
kosli attest custom \
--type=coverage-metrics \
--attestation-data=unit-test-coverage.json \
...
```

Expand All @@ -349,6 +349,10 @@ Currently, we support the following types of evidence:
- So `32 / 1209 * 100 <= 5` evaluates to `2.64 <= 5` which is `true`


A custom attestation type can also define a **summary**, so that attestations of the type show
named values from their data instead of only raw JSON. See
[Summarizing custom attestations](#summarizing-custom-attestations) below.

See:
* [create custom attestation type](/client_reference/kosli_create_attestation-type) and
* [report custom attestation to an artifact or a trail](/client_reference/kosli_attest_custom/) for usage details and examples.
Expand All @@ -364,3 +368,58 @@ Currently, we support the following types of evidence:
</Accordion>
</AccordionGroup>

## Summarizing custom attestations

By default, the attestation detail page in Kosli shows a custom attestation type's evaluation rules
as a pass/fail checklist, plus the raw attestation data. Add a **summary** to the type to pull named
values out of the attestation data and display them as labeled rows instead — the way the built-in
Sonar, Snyk and JUnit attestations do.

A summary is an ordered list of `name`/`expression` pairs, where each expression is a
[jq expression](https://jqlang.org/manual/) evaluated against the attestation data. Add one entry
per `--summary` flag:

```bash
kosli create attestation-type coverage-metrics \
--jq=".code.lines.missed / .code.lines.total * 100 <= 5" \
--summary="Lines missed=.code.lines.missed" \
--summary="Lines total=.code.lines.total"
```

Attestations of the `coverage-metrics` type then show `Lines missed` and `Lines total`, in that
order, with the values taken from each attestation's own data. Re-running
`kosli create attestation-type` for a name that already exists updates that type rather than
creating a second one — see the note on versioning below.

Each `--summary` value is split on its first `=` only, so `==` inside a jq expression is safe. Use
[`--summary-json`](/client_reference/kosli_create_attestation-type) instead if the list is easier to
express as JSON, or the
[`summary` attribute](/terraform-reference/resources/custom_attestation_type) if you manage your
types with Terraform:

```json
[
{ "name": "Lines missed", "expression": ".code.lines.missed" },
{ "name": "Lines total", "expression": ".code.lines.total" }
]
```

Worth knowing:

- The summary is part of the **versioned** type definition. Changing it creates a new version of the
type, just like changing the schema or the evaluation rules. Each attestation is displayed using
the version of the type it was reported against, so editing a summary does not change how existing
attestations render.
- Expressions are evaluated when the attestation is **displayed**, against the stored attestation
data. A summary never affects the compliance status of an attestation.
- Invalid jq is rejected when the type is created, so authoring mistakes surface immediately rather
than as a broken detail page later.
- An expression that returns `null`, or that fails against a particular attestation's data, renders
as `N/A`. The rest of the summary still renders.
- A value renders as a clickable link only when it is a string beginning with `http://` or
`https://`. Everything else — including URLs with another scheme — renders as text. This is
stricter than the rule for [annotation values](#annotating-attestations).
Comment thread
dangrondahl marked this conversation as resolved.
- If the attestation data is a top-level array, Kosli renders one summary group per element,
numbering them (`Summary 1:`, `Summary 2:`, and so on) when there is more than one. Write the
expressions against a single element (`.code.lines.missed`) — each element is evaluated
separately.
10 changes: 8 additions & 2 deletions tutorials/attest_custom.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,20 @@ Before you can report a custom attestation, the type referenced by `--type` must
* **CLI** — [`kosli create attestation-type`](/client_reference/kosli_create_attestation-type) (good for quick experiments).
* **Terraform** — the [`kosli_custom_attestation_type` resource](/terraform-reference/resources/custom_attestation_type) (recommended so the type is version-controlled).

For this tutorial we'll create a minimal `coverage-report` type that requires a `coverage` field of at least 80:
For this tutorial we'll create a small `coverage-report` type that requires a `coverage` field of at least 80:

```shell
kosli create attestation-type coverage-report \
--description "Code coverage report" \
--jq '.coverage >= 80'
--jq '.coverage >= 80' \
--summary "Coverage=.coverage" \
--summary "Tool=.tool"
```

The `--summary` entries are optional. Each is a `'NAME=EXPRESSION'` pair, so the attestation you
report below opens on a readable `Coverage: 92` / `Tool: pytest-cov` summary instead of raw JSON —
see [Summarizing custom attestations](/getting_started/attestations#summarizing-custom-attestations).

Prepare a JSON file with the data you want to attest. Save it as `coverage.json`:

```json
Expand Down