Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions calico-enterprise/threat/gateway-waf/troubleshooting.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -181,6 +181,8 @@ For how to write exception rules within your namespace's ID range, see [Write cu

:::

When the Core Rule Set misfires on a known application — for example, Kibana Discover returning `403` because the Core Rule Set reads its Elasticsearch query bodies as attacks — tune it with a cluster-scoped `GlobalWAFPlugin`, which may adjust how the Core Rule Set runs. A `GlobalWAFPlugin` loads on every WAF-covered route in the cluster, so pin its rules to the target host and path. See [Unblock Kibana Discover](./write-custom-rules.mdx#example-unblock-kibana-discover).

---

## General diagnostic sequence
Expand Down
92 changes: 92 additions & 0 deletions calico-enterprise/threat/gateway-waf/write-custom-rules.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,98 @@

Gateway WAF runs the [OWASP Core Rule Set v4](https://coreruleset.org/docs/) via [Coraza](https://coraza.io/); the [ModSecurity reference manual](https://github.com/owasp-modsecurity/ModSecurity/wiki/Reference-Manual-(v3.x)) covers the full SecLang directive set.

## Example: unblock Kibana Discover

Kibana sends an Elasticsearch query in the body of its search requests, and a `url` argument on many requests. With the Core Rule Set enabled, the WAF reads the search body as an attack and the `url` argument as server-side request forgery, so Kibana Discover fails with HTTP 403. The Core Rule Set is working as designed — Kibana needs app-specific exceptions, the same way the Core Rule Set ships exceptions for applications such as phpMyAdmin and WordPress.

These exceptions change how the Core Rule Set runs, so they belong in a `GlobalWAFPlugin`. A namespaced `WAFPlugin` cannot alter a baseline rule — the admission webhook rejects it with `ProtectedIDRemoval` — and a namespaced `WAFPolicy` cannot reference a `GlobalWAFPlugin`. A cluster-scoped `GlobalWAFPlugin`, loaded through the `default` `GlobalWAFPolicy`, is the only configuration that both tunes the Core Rule Set and reaches the gateway.

A Discover search carries a full Elasticsearch query in its body — sort fields, highlight tags, field lists, nested filters, and large numbers. The Core Rule Set cannot tell this query apart from an attack: many of its rules each match some part of the body and add to an anomaly score, and rule `949110` blocks the request once that score crosses the threshold. The set of rules that match changes from query to query, because each search sends different fields and values. Removing one rule, or even the whole SQL-injection group, does not reliably unblock Discover — another rule brings the score back over the threshold on the next query.

So the exception skips Core Rule Set inspection of the request **body**, instead of removing named rules — and it matches as narrowly as it can: only a `POST`, to the Manager gateway host, on the exact search path. The WAF still inspects the URL, headers, and method on that request, still inspects the body on every other path, and the exception never applies on another gateway that serves the same path. The `url` argument is different: it rides in the URL, has a fixed name, and trips two specific rules, so its exception removes just that argument from those two rules — pinned to the same host.

| Request | What fires | Exception |
| :--- | :--- | :--- |
| `POST /tigera-kibana/internal/bsearch` on the Manager host (Discover search) | The search body scores across several Core Rule Set rules and crosses the anomaly threshold (`949110`) | Skip request-body inspection for that one request |
| `/tigera-kibana/...` requests with a `url` argument, on the Manager host | `931100`, `934110` read the `url` argument as server-side request forgery | Remove the `url` argument from those two rules |

Apply the plugin and reference it from the `default` `GlobalWAFPolicy`, which every `WAFPolicy` reads:

```yaml
apiVersion: applicationlayer.projectcalico.org/v3
kind: GlobalWAFPlugin
metadata:
name: kibana-exceptions
spec:
description: "Core Rule Set exceptions for Kibana on the Manager gateway"
before: |
# Discover sends its Elasticsearch query in the body of a POST to one search
# path. The Core Rule Set scores that body as an attack, so skip request-body
# inspection for exactly that request: on the Manager gateway host, a POST to
# the exact search path. URL, header, and method inspection stay in force.
# Replace manager.example.com with your Manager gateway hostname
# (Manager spec.ingressGateway.hostname). SERVER_NAME is the gateway host from
# the :authority header. The ctl sits on the last rule of the chain, not the
# first, so it only runs when the whole chain matches. The chained rules carry
# no id — the admission check rejects a chained rule that has one.
SecRule SERVER_NAME "@streq manager.example.com" \
"id:100000001,phase:1,pass,nolog,t:none,t:lowercase,chain"
SecRule REQUEST_METHOD "@streq POST" \
"t:none,chain"
SecRule REQUEST_FILENAME "@streq /tigera-kibana/internal/bsearch" \
"t:none,t:urlDecodeUni,t:normalizePath,ctl:requestBodyAccess=Off"
# The url argument reads as server-side request forgery. On the Manager host
# only, remove that one argument from the two forgery rules.
SecRule SERVER_NAME "@streq manager.example.com" \
"id:100000002,phase:1,pass,nolog,t:none,t:lowercase,chain"
SecRule REQUEST_FILENAME "@beginsWith /tigera-kibana/" \
"t:none,t:urlDecodeUni,t:normalizePath,\
ctl:ruleRemoveTargetById=931100;ARGS:url,\
ctl:ruleRemoveTargetById=934110;ARGS:url"
---
apiVersion: applicationlayer.projectcalico.org/v3
kind: GlobalWAFPolicy
metadata:
name: default
spec:
plugins:
- kind: GlobalWAFPlugin
name: kibana-exceptions
Comment on lines +189 to +191

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A namespaced WAFPolicy can't reference a GlobalWAFPlugin (EV-6753) and only a global plugin can tune a CRS baseline rule, so the default GlobalWAFPolicy merge is the only viable wiring — cluster-wide scope now documented. Fixed in the latest push.

```

If the cluster already has a `default` `GlobalWAFPolicy`, do **not** re-apply the block above — a second `kubectl apply` drops any field the file omits, so a `defaultAction: Block` would fall back to `Detect` for every policy that inherits it. Append the plugin with a patch instead:

```bash
kubectl patch globalwafpolicy default --type=json \
-p='[{"op":"add","path":"/spec/plugins/-","value":{"kind":"GlobalWAFPlugin","name":"kibana-exceptions"}}]'
```

Apply the full `GlobalWAFPolicy` block above only on a cluster that has no `default` yet.

:::caution

Trade-off: on the Manager gateway host, the WAF does not inspect the **body** of a Discover search (the `POST` to `/tigera-kibana/internal/bsearch`). Its URL, headers, and method are still inspected, and every other request, path, and host is inspected in full.

:::

:::note

The plugin **loads** on every WAF-covered gateway and route: the controller merges `GlobalWAFPolicy/default` into every `WAFPolicy` in the cluster. The `SERVER_NAME` host match and the path match keep it **firing** only for Kibana on the Manager gateway, so a second gateway serving the same paths is unaffected. Set the host in both rules to your Manager gateway hostname.

:::

:::note

There is no namespaced way to drop a single Core Rule Set rule — a `WAFPlugin` cannot remove or retarget a baseline rule (IDs `900000`–`999999` or the Tigera-managed range). The nearest namespaced alternative is a `WAFPolicy` with `action: Detect`, in the Manager gateway's namespace (`calico-system`, unless `Manager.spec.ingressGateway.gatewayNamespace` says otherwise), targeting the `calico-manager-route` `HTTPRoute`. The cost is that it turns the **whole** Manager gateway to Detect — it stops blocking for the entire Manager UI, not just Kibana — because the operator renders that route as one rule with no path match and reverts hand edits. Use the `GlobalWAFPlugin` above if you need to keep blocking everywhere else.

Check failure on line 217 in calico-enterprise/threat/gateway-waf/write-custom-rules.mdx

View workflow job for this annotation

GitHub Actions / runner / vale

[vale] reported by reviewdog 🐶 [Vale.Spelling] Did you really mean 'retarget'? Raw Output: {"message":"[Vale.Spelling] Did you really mean 'retarget'?","location":{"path":"calico-enterprise/threat/gateway-waf/write-custom-rules.mdx","range":{"start":{"line":217,"column":99},"end":{"line":217,"column":107}}},"severity":"ERROR","code":{"value":"Vale.Spelling"}}

:::

:::note

The rule IDs `100000001` and `100000002` are safe to reuse and to copy for other apps: namespace ID windows stop at `99,999,999` and the protected ranges (Core Rule Set and Tigera baseline) stop at `1,099,999`, so IDs from `100,000,000` up can't clash with a tenant's rules or a baseline rule. The search endpoint can differ between Kibana versions — read the `tigera_secure_ee_waf` log index for your cluster to confirm the search path, and add a rule per search endpoint rather than loosening the exact-path match. Because the body exception keys on the request (`SERVER_NAME` + `POST` + path) and not on named rules, it keeps working as the Core Rule Set changes which rules match.

:::

## Next steps

- [Validate configuration with Rego](./validate-config.mdx) — enforce guardrails on the merged ruleset.
Expand Down
Loading