zuumb

An AI-Native Detection & Response (AIDR) layer on top of Wazuh: Wazuh alert → LLM triage verdict → entity/time correlation into incidents → attack chain, response-task proposals, analyst feedback loop, dashboard.

Built as the next layer on top of prior Wazuh detection-rule work.

Correlation itself is two layers, not one: a deterministic entity/time engine that's always on and actually forms incidents, plus a separate, advisory local embedding model that flags incidents which read as the same campaign even with no shared host, IP, or user — surfaced as a hint, never authoritative. See Finding related incidents that share nothing obvious.

Demo

A minute on the real dashboard, the real reasoning behind a verdict, a real attack chain, and exactly where a language model is and isn't involved. Click the image to play the video.

How it fits together

Three views of the same system, drawn from the code rather than from memory. Click any image for the interactive version, where every box carries a SRC badge linking to the exact source lines it was drawn from, pinned to commit 65b1ecb. The interactive pages need nothing installed; they're single self-contained HTML files in assets/diagrams/.

System architecture

Who talks to whom. The main path runs left to right along the middle row; the rest hangs off it. Above the triage agent, secret redaction sits between the alert and the one LLM call. Below it are the local second-opinion classifier and its weekly retrain loop. Up and to the right of correlation sit the advisory similarity layer and, above it, the human-confirmed manual links. On the response side, playbooks propose tasks, a person approves them at the approval gate, and only then, outside dry-run, does Wazuh Active Response get called. Stages hand off through the database rather than calling each other.

One pipeline cycle

What happens each time the poller fires, and what only happens when a person is involved. The live poller is opt-in (WAZUH_LIVE_POLLING=true) and runs every 60 seconds by default. Each cycle ingests new alerts and triages up to 40 untriaged ones; an alert that fails three times is quarantined. Only if something new came in does it rebuild incidents, chains, and similarity. Response tasks are not part of the cycle: they are proposed when an analyst opens an incident, and nothing dispatches until a person approves it.

Information flow

What each piece of data is and where it travels: raw Wazuh alert JSON, then Alert rows, then a masked brief, then a verdict, then Incident rows, which fan out into chains, similarity hints, and proposed tasks. The dashed paths are advisory or learning loops: analyst overrides come back as few-shot examples for the LLM and as training data for the weekly second-opinion retrain.

A few things the diagrams leave out on purpose, so the picture stays readable: every stage also reads and writes the shared database, correlation reads Alert rows for entities and timestamps as well as verdicts, the manual-links layer appears only in the architecture view because it is display-only, and detector D2 (network beacons) is a manual CLI, not a running service.

Supported versions

Wazuh	Status
4.9.x	Fully supported. The alert normaliser targets the Wazuh 4.x schema and is tested against a live 4.9.2 single-node stack.
5.x / OCSF	Not yet supported. The 5.x ECS/OCSF event schema differs from the 4.x schema the normaliser targets. Support is tracked as future work (build plan §11).

Prerequisites

• Python 3.11+
• An Anthropic API key (platform.claude.com) — the only paid dependency
• For live polling only: Docker and a Wazuh 4.9.x single-node stack (setup below)

Quickstart

With Docker (no local build)

git clone https://github.com/NucleiAv/zuumb && cd zuumb
cp .env.example .env            # fill in ANTHROPIC_API_KEY and the WAZUH_* creds
docker compose up -d            # pulls ghcr.io/nucleiav/zuumb:latest

Open http://localhost:8000.

• The SQLite database lives in a named volume (zuumb-data); nothing else to set up.
• Set a dashboard password. With DASHBOARD_PASSWORD blank the UI and the /tasks/*/approve endpoint are open to anyone who can reach the port; startup logs a warning. Set it (and SESSION_SECRET to any long random string) before the container is reachable by another browser. Then sign in at /login.
• Wazuh runs elsewhere. zuumb reads an existing Wazuh stack — it doesn't start one. In .env, point every WAZUH_*_API_URL at host.docker.internal instead of localhost (the compose file already maps that name to the host).
• Active response ships in dry-run (RESPONSE_DRY_RUN=true): approving a response task records what would have run and dispatches nothing. Only set it to false once you understand it will run real firewall-drop / disable-account commands on your agents — see Automated mitigation.
• Contributors who want to build from source and live-reload: docker compose -f docker-compose.dev.yml up --build.

From source

cp .env.example .env           # then put your ANTHROPIC_API_KEY in .env

python -m venv .venv
. .venv/Scripts/activate       # Windows;  ".venv/bin/activate" on macOS/Linux
pip install -r requirements.txt
pytest -q

Run it on synthetic Wazuh alerts — no live Wazuh, no Docker, SQLite is created automatically:

python -m scripts.run_poc [--offline]
.venv/Scripts/python -m uvicorn app.main:app --reload   # http://localhost:8000

For a live feed, see Live Wazuh below, then set WAZUH_LIVE_POLLING=true in .env — app.main starts a background poller that runs poll → triage → correlate → stitch on a schedule.

Live Wazuh (4.9.x)

zuumb does not ship Wazuh; it reads alerts from an existing stack's indexer (wazuh-alerts-*, port 9200 — not the Manager API on 55000). This is the exact flow used to stand one up under WSL Docker and enrol an agent.

Docker here runs under WSL, so commands are docker … inside WSL; ports are published to localhost on the Windows host.

1. Wazuh 4.9.2 single-node stack

git clone --depth 1 -b v4.9.2 https://github.com/wazuh/wazuh-docker.git ~/wazuh-docker-4.9.2
cd ~/wazuh-docker-4.9.2/single-node

docker compose -f generate-indexer-certs.yml run --rm generator

# fresh indexer volume, owned by the indexer UID — skips a node.lock
# AccessDenied crash loop, and keeps this stack's volumes separate from any
# other wazuh-docker project on the machine (hence COMPOSE_PROJECT_NAME).
docker volume create wazuh49_wazuh-indexer-data
docker run --rm -v wazuh49_wazuh-indexer-data:/data alpine \
  sh -c 'chown -R 1000:1000 /data && chmod -R 770 /data'

COMPOSE_PROJECT_NAME=wazuh49 docker compose up -d

# wait for green
curl -sk -u admin:SecretPassword https://localhost:9200/_cluster/health

Defaults from this deployment: indexer admin / SecretPassword, Manager API wazuh-wui / MyS3cr37P450r.*-, dashboard on https://localhost:443.

2. Read-only ingestion user

zuumb should read as a least-privilege user, never admin. Create one on the indexer's OpenSearch security API:

ADMIN='admin:SecretPassword'
ING_PASS='choose-a-strong-password'

curl -sk -u "$ADMIN" -X PUT "https://localhost:9200/_plugins/_security/api/roles/zuumb_ingest_ro" \
  -H 'Content-Type: application/json' \
  -d '{"cluster_permissions":["cluster_composite_ops_ro"],"index_permissions":[{"index_patterns":["wazuh-alerts-*"],"allowed_actions":["read","search","indices:admin/mappings/get","indices:monitor/settings/get"]}]}'

curl -sk -u "$ADMIN" -X PUT "https://localhost:9200/_plugins/_security/api/internalusers/zuumb-ingest" \
  -H 'Content-Type: application/json' -d "{\"password\":\"$ING_PASS\"}"

curl -sk -u "$ADMIN" -X PUT "https://localhost:9200/_plugins/_security/api/rolesmapping/zuumb_ingest_ro" \
  -H 'Content-Type: application/json' -d '{"users":["zuumb-ingest"]}'

# verify: read is allowed, write is refused
curl -sk -o /dev/null -w 'read  %{http_code} (want 200)\n' -u "zuumb-ingest:$ING_PASS" \
  "https://localhost:9200/wazuh-alerts-*/_search?size=1"
curl -sk -o /dev/null -w 'write %{http_code} (want 403)\n' -u "zuumb-ingest:$ING_PASS" \
  -X POST "https://localhost:9200/zuumb-probe/_doc" -H 'Content-Type: application/json' -d '{}'

3. Enrol an agent

Run on whatever host you want telemetry from. Here it is the WSL Ubuntu box itself:

curl -so /tmp/wazuh-agent.deb \
  https://packages.wazuh.com/4.x/apt/pool/main/w/wazuh-agent/wazuh-agent_4.9.2-1_amd64.deb

sudo WAZUH_MANAGER='localhost' WAZUH_AGENT_NAME='my-agent' dpkg -i /tmp/wazuh-agent.deb
sudo systemctl daemon-reload
sudo systemctl enable --now wazuh-agent
sudo /var/ossec/bin/wazuh-control status

WAZUH_MANAGER + the first service start auto-enrols against the manager's authd on port 1515 (published by the compose stack). Confirm the manager sees it:

docker exec wazuh49-wazuh.manager-1 /var/ossec/bin/agent_control -l

4. Confirm alerts are landing

# make a little noise
for i in 1 2 3; do sudo -k; echo wrongpass | sudo -S true 2>/dev/null; done
sudo cat /etc/shadow > /dev/null

curl -sk -u admin:SecretPassword "https://localhost:9200/_cat/indices/wazuh-alerts-*?v"

A wazuh-alerts-4.x-* index with a non-zero docs.count means the feed is live.

5. Point zuumb at it

# .env
WAZUH_API_URL=https://localhost:9200
WAZUH_API_USER=zuumb-ingest
WAZUH_API_PASSWORD=the-password-you-chose
WAZUH_VERIFY_SSL=false
WAZUH_ALERTS_INDEX=wazuh-alerts-*
WAZUH_LIVE_POLLING=true
WAZUH_POLL_SECONDS=60

.venv/Scripts/python -m uvicorn app.main:app --host 0.0.0.0 --port 8000

The console logs each cycle (wazuh poll: ingested=… triaged=…). The first run triages the whole backlog in batches of 40 per tick, so incidents and charts fill in progressively — watch http://localhost:8000.

Postgres (optional, replaces SQLite): docker compose up -d postgres, then set DATABASE_URL in .env.

Adding Wazuh agents

Attack chains only form when activity spans more than one source, so zuumb wants several agents reporting. Two ways to add them.

A. Container agents (the multi-host lab)

There's no wazuh/wazuh-agent image for 4.9.x (Docker Hub starts at 4.13), so docker/agent/ builds one from the official .deb. docker-compose.agents.yml runs two of them.

Run every docker compose -f docker-compose.agents.yml … command from the repo root (the file is ./docker-compose.agents.yml). From WSL that's cd /mnt/<drive>/…/zuumb first. Plain docker … commands (below) work from anywhere.

# the Wazuh stack must be up first (it owns the network the agents join)
docker compose -f docker-compose.agents.yml up -d --build

• agent-lab-01, agent-lab-02 join the manager's wazuh49_default network and enrol automatically on first start.
• Each keeps its enrolment key in a volume (agentNN-etc). The manager rejects a duplicate agent name, so an agent must enrol exactly once — the volume lets restart / recreate reconnect instead of re-enrolling. To force a clean re-enrol: docker compose -f docker-compose.agents.yml down -v.
• restart: unless-stopped — they come back on every Docker / laptop restart until you explicitly docker compose -f docker-compose.agents.yml down.
• LAB_NOISE=1 (in the compose file) seeds realistic sshd auth-failure traffic with rotating source IPs so the pipeline has data while real traffic builds. Remove LAB_NOISE when the agents monitor real hosts — a real deployment watches real activity, it doesn't manufacture it.

Add a third (Nth) agent — in docker-compose.agents.yml, copy a service block and add its volume:

services:
  agent-lab-03:
    <<: *agent
    hostname: agent-lab-03
    volumes: ["agent03-etc:/var/ossec/etc"]
volumes:
  agent03-etc:

then docker compose -f docker-compose.agents.yml up -d --build. The hostname becomes the agent name the manager registers.

Container agent commands — from the repo root:

docker compose -f docker-compose.agents.yml ps            # status / health
docker compose -f docker-compose.agents.yml logs -f agent-lab-01
docker compose -f docker-compose.agents.yml restart agent-lab-02
docker compose -f docker-compose.agents.yml down          # stop + remove all
docker compose -f docker-compose.agents.yml build --no-cache   # rebuild the image (e.g. new .deb)

…or from anywhere, by container name (docker ps to list them):

docker ps --filter name=zuumb-agents            # the agent containers + health
docker restart zuumb-agents-agent-lab-02-1
docker logs -f zuumb-agents-agent-lab-01-1

B. A real host (bare-metal or VM)

Same steps as Live Wazuh → 3. Enrol an agent above — run them on the other machine. Set WAZUH_MANAGER to localhost only if that machine runs the stack, otherwise to the manager host's LAN IP, and give each host a distinct WAZUH_AGENT_NAME. Windows/macOS use the platform installer from https://packages.wazuh.com/4.x/ with the same two env vars.

How many agents are running

# canonical count — active / disconnected / never_connected / total (excludes the manager itself)
TOKEN=$(curl -sk -u wazuh-wui:'MyS3cr37P450r.*-' -X POST \
  "https://localhost:55000/security/user/authenticate?raw=true")
curl -sk -H "Authorization: Bearer $TOKEN" "https://localhost:55000/agents/summary/status"
# -> {"data":{"connection":{"active":3,"disconnected":0,"never_connected":0,"pending":0,"total":3}, ...}}

# quick check without the API token (counts id 000, the manager, too — subtract 1)
docker exec wazuh49-wazuh.manager-1 /var/ossec/bin/agent_control -l | grep -c Active

# just the container agents (from the repo root)
docker compose -f docker-compose.agents.yml ps

Verify an agent

# manager's view — the agent should read Active, not "Never connected" / Disconnected
docker exec wazuh49-wazuh.manager-1 /var/ossec/bin/agent_control -l

# its alerts are reaching the indexer
curl -sk -u admin:SecretPassword \
  "https://localhost:9200/wazuh-alerts-*/_search?size=3&q=agent.name:agent-lab-01&sort=timestamp:desc"

If an agent is stuck "Never connected": it enrolled (has a key) but can't reach the manager on 1514/tcp — check the address it's using (grep '<address>' /var/ossec/etc/ossec.conf in the agent) and that 1514 is published by the stack.

If a container agent logs Duplicate agent name … Unable to add agent: its key volume is out of sync with the manager. docker compose -f docker-compose.agents.yml down -v then up to re-enrol from scratch.

Remove an agent

docker compose -f docker-compose.agents.yml stop agent-lab-02      # container: stop it
docker exec wazuh49-wazuh.manager-1 /var/ossec/bin/manage_agents -r <id>   # id from agent_control -l

The stack's authd has <purge>yes</purge>, so a removed key is cleared from the manager automatically on the next restart.

Daily bring-up

After a full shutdown (otherwise everything auto-restarts):

wsl bash scripts/lab-up.sh     # Wazuh stack -> wait for indexer -> agents -> prints agent list
# then, in the repo:
.venv/Scripts/python -m uvicorn app.main:app --host 0.0.0.0 --port 8000

What's rule-based, what's learned, and what's a language model

Worth being precise about this, since "AI detection" gets used loosely enough elsewhere that it stops meaning anything.

Wazuh's own alerts are rule-based, signature and threshold matching, exactly what Wazuh always did, unchanged by anything zuumb adds on top.

Two pieces zuumb adds are machine learning, not language models, small, deterministic given a fixed model, and running entirely on the machine with no API call and no per-alert cost. The auth-log anomaly detector fits a statistical baseline per host and flags a window that doesn't look like it. The second-opinion classifier is a simple linear model over word patterns in the alert text. The incident-similarity signal is also local and model-based, a small sentence-embedding model rather than a classifier, still not a language model, no free-form text, just a similarity number between two incidents.

Exactly one piece in the whole pipeline is a large language model, the primary triage verdict, the one that reads an alert and writes a plain-language reason for its call. That's the part with real per-call cost and the part that can be turned off from the nav bar, at which point even that step falls back to the deterministic second-opinion classifier instead.

How zuumb detects things on its own

Wazuh already finds a lot by itself. zuumb now also runs two small engines of its own that watch for patterns Wazuh's rules were never written to catch, and feeds whatever they find through the exact same pipeline every other alert goes through (reserved rule.id ranges, so you can always tell where an alert came from, 900001 to 900999 for the first engine, 901001 to 901999 for the second).

The first one watches ssh login activity. It pulls the raw auth log lines Wazuh already collected, groups them into five minute windows per host, and learns what a normal window looks like for that host. When a window looks statistically off, say a sudden burst of failed logins, it raises its own alert. This one runs continuously. A small container polls the Wazuh alert index every ten minutes on its own, nothing to run by hand.

python -m detectors.authlog.run --log /var/log/auth.log   # run it ad hoc against a file instead

The second looks for something quite different, network beacons. That's when a machine calls out to the same destination over and over on an almost perfectly regular schedule, the classic sign of something phoning home to a command and control server. This one only runs when you point it at a real file of connection records, something like a Zeek conn.log.

python -m detectors.netflow.run --conn <file>

It isn't scheduled to run continuously because there's nothing feeding it live network data yet. Wazuh doesn't capture that kind of traffic on its own, and building a fake source just to keep a container busy would defeat the point of the whole exercise, so it stays a manual tool until a real network sensor exists.

Because a scheduled job can quietly stop working with nobody noticing, the incidents page shows a small line near the top saying how long ago each continuous detector last checked in. If it hasn't run in a while, that line turns red, so a dead job is obvious rather than assumed fine.

Teaching the second opinion from real corrections

Every alert zuumb triages gets a verdict from the main Claude model, plus a second, much cheaper opinion from a small local model running right there on the machine, no API call needed. It exists purely as a sanity check. If the two disagree, and the local model thinks something is worse than what Claude said, that gets flagged for a human to glance at. Agreement, or a milder local opinion, is treated as noise and stays quiet.

At first this local model only knows what it learned from the hand written example alerts in the eval set. But every time an analyst corrects a verdict in the dashboard, that correction becomes a real training example (AnalystFeedback). Once a week, a background process gathers whatever corrections have piled up, retrains the local model on them, and checks its accuracy against a fixed set of held out examples before deciding whether to actually put the new version to work.

python -m app.triage.retrain            # run one retrain pass by hand
python -m app.triage.retrain --live     # the loop the container runs, weekly

If the new version isn't at least as good as the one already running, it gets thrown away and the old one keeps serving. Nothing about this loop can let the model quietly get worse over time, and the very first run always goes live so the whole thing isn't sitting silently empty before any feedback shows up.

The incidents page shows which version is live, how many real corrections it learned from, how it scores on the held out set, and a rough read on how different recent alerts look compared to what it trained on. A big drop in that last number is worth a look, it can mean the model's gone stale, but nothing acts on it automatically. It's information for a person, same as everything else in zuumb.

Testing the second opinion against a deliberate attack

A model that scores well on a clean eval set has never actually been tested against someone trying to fool it, which is a different question. So the second-opinion classifier got run through the Adversarial Robustness Toolbox, crafting the smallest possible nudge to every alert it currently calls malicious or suspicious correctly, aimed specifically at flipping the prediction to benign, the one flip an attacker actually wants.

The finding is that the decision boundary has almost no room in it. A nudge of about a hundredth of a typical feature's own value is enough to flip every one of those alerts to benign. That's a small model over a small vocabulary, so a thin margin isn't shocking, but it hadn't actually been measured before.

python -m eval.adversarial

Worth being precise about what this does and doesn't show. It proves the model's decision boundary is fragile in its own numeric feature space. It doesn't hand over a rewritten alert that actually evades it, turning a perturbation vector into real words to add to a log line is a separate, harder problem nobody's attempted here. Full numbers and reasoning are in eval/ADVERSARIAL.md. This doesn't change anything about how the classifier is used today, it's advisory only, but it matters more for the one mode where this classifier becomes the primary verdict, see the next section.

Turning AI detection off

There's a switch in the top nav bar, labeled AI ON or AI OFF, that any logged in analyst can click. It's one setting for the whole deployment, not a personal preference, since triage runs once in the background as alerts come in rather than once per person looking at the dashboard.

With it on, every new alert goes through the normal path, the Claude model reads it and writes a verdict with its reasoning in plain language. Click it off and that call stops happening entirely, no API cost, no LLM in the loop at all. Alerts still get triaged automatically though, just by the small local classifier described above standing in as the primary verdict instead of a cross check. It won't have a written explanation, only a label and a confidence number, and the dashboard says plainly where that verdict came from so nobody mistakes it for the LLM's reasoning.

Flip it back on and the next alert that comes in goes straight back through Claude, nothing needs a restart.

Finding related incidents that share nothing obvious

Correlation groups alerts by shared host, IP, or user, which catches most real intrusions but misses a real shape too, the same campaign spread across machines with nothing literal in common between them. Nothing in the deterministic engine can see that, by construction, since it's only looking for a matching identifier.

So there's now a second, entirely separate signal for it. A small local sentence-embedding model reads what each incident's alerts actually say, ignoring host, IP, and user on purpose, and flags a pair as worth a look when they read as the same kind of activity despite sharing no entity at all. It never runs on a pair that already shares an entity, since correlation or the chain stitcher already have that covered, and flagging it twice would just be noise.

This never merges incidents, never changes severity, and never feeds back into correlation or chain stitching. It shows up as one line on the incident detail page, something like "might be the same campaign as incident #42," and that's the whole extent of what it does. Same trust posture as the second opinion, a nudge for a human, nothing more.

Confirming a link by hand

The similarity hint is a suggestion, so the incident detail page also lets an analyst turn one into an explicit record: a "confirm this is related" action on the banner writes who confirmed it, when, and an optional note (e.g. "confirmed via manual log review, same attacker IP range not captured in our entity fields").

This is a third, separate layer — not the deterministic correlation, not the embedding similarity, and not an attack chain. It never feeds into any of them: no change to severity, no AttackChain created or modified, and confirming a pair that was never flagged similar in the first place works the same way. It's shown in its own "Analyst-linked incidents" section, labeled human-confirmed and styled distinctly from both the advisory banner and a chain view, so it can never read as something zuumb detected on its own. Confirming the same pair twice is a no-op, not a duplicate row, and an analyst can unlink it later — a past judgment call isn't permanent.

Attack chains — what they are and aren't

An attack chain is a grouping hypothesis, not a proven forensic timeline or attacker attribution. zuumb takes medium/high-severity incidents that share a host, IP, or user within a time window, orders them by MITRE ATT&CK tactic, and calls that a chain. It does not verify that one stage caused the next, and the tactic ordering is by kill-chain phase, not by observed progression. A Monday recon scan and an unrelated Thursday file transfer on the same host are not one attack, even though the grouping would put them in sequence.

To keep a coincidence from reading like a real multi-stage attack, every chain carries a confidence score (high / medium / low), computed at build time and shown on the chains list and detail pages. It is downgraded for: a single shared host being the only link (jump-box shape), stages linked only transitively, a large time gap between linked stages, tactic labels guessed from technique ids rather than taken from Wazuh's own mapping, or the whole chain resting on one shared entity. A low-confidence chain shows an unmissable warning on its detail page. Analysts can additionally mark a chain confirmed-narrative or false-chain — that human causation call is separate from the system's confidence score.

Treat any chain as a lead to review, not a conclusion. The eval-numbers framing applies here too: directional, not certified.

Tuning on real traffic

Let live data accumulate for a few days, then run the CLI diagnostic:

python -m scripts.chain_quality

It prints, per chain, its confidence, the entities joining each adjacent stage pair, and a flag for the two false-chain shapes the confidence score penalises:

• hub-host — every link is one shared host (a proxy / jump box, not an attack path)
• weak-link — adjacent stages share no entity directly (stitched only transitively)

Knobs (all .env-overridable):

var	default	when to change
CORRELATION_WINDOW_MINUTES	30	shrink if noisy traffic over-merges unrelated alerts into one incident
CHAIN_MAX_ENTITY_SPREAD	4	lower if a busy shared host keeps stitching unrelated incidents together
CHAIN_STRONG_LINK_HOURS	24	stages linking within this gap keep full confidence
CHAIN_MAX_LINK_HOURS	72	beyond this gap, incidents don't link into a chain at all

Five questions for any "AI-powered" claim

A useful gut-check for any product that says "AI-powered," including this one: ask these five questions, and if the answers are vague, that's the actual finding. Here's zuumb's, answered plainly.

1. What type of AI? Four different things, not one. Wazuh's own alerts are rule-based, signature matching, unchanged. The auth-log anomaly detector (D1) is classical unsupervised ML, PyOD's ECOD scoring windows against a learned per-host baseline. The network-beacon detector (D2) isn't ML at all, it's a deterministic statistics check, coefficient of variation on connection timing, worth saying plainly rather than dressing it up. The second-opinion classifier (D3) is classical supervised ML, TF-IDF plus logistic regression, trained on real analyst corrections. The incident-similarity signal is a small local deep-learning model, a sentence embedding, still not a language model. And exactly one piece is a large language model: Claude writes the primary triage verdict and its reasoning. A nav-bar toggle turns that one piece off; everything else keeps running.

2. What metric, on what data, vs. what baseline? Triage accuracy, currently about 89%, against a 104-alert hand-labeled set, is the headline number, and it comes with real limits worth stating rather than burying: a small, self-labeled, synthetic set, not independently sized or class-balanced against a real deployment's base rate, and there's no published comparison against a simpler baseline like keyword matching. D3's own retrain loop holds a higher bar — every retrain is scored against a fixed held-out split and only promoted if it actually beats the live model, never just for being newer. D1 and D2 have no accuracy number at all; they're unsupervised and heuristic, with no labeled ground truth to score them against. That's a known gap, not an oversight.

3. Who evaluated it? One person, the one who built it. Every number in this repo is self-reported and run internally — no independent lab, no peer review, no external red team. This is a portfolio and demo project, not a vendor claim, but the honest answer to this question doesn't change because of that.

4. What are the failure modes? Specific ones, not a hand-wave. Turning the LLM off quietly downgrades triage quality to the local classifier, a real tradeoff, not a free lunch. The second-opinion classifier has a genuinely thin margin around its decision boundary — a small adversarial test found a crafted alert can flip it (eval/ADVERSARIAL.md). The auth-log detector needs a clean baseline period; a host that's noisy from day one has nothing normal to compare against. The beacon detector only catches near-perfectly-regular timing by design — a beacon that jitters its interval on purpose slips through, an accepted blind spot of that technique in general, not a bug in this one. The correlation engine only looks at shared host, IP, or user — a campaign spread across machines with nothing in common on paper is invisible to it unless the advisory similarity layer happens to flag it. An attack chain is a grouping hypothesis, not causation — two unrelated things on the same busy jump box can get stitched together. And a human-confirmed manual link is a person's opinion, recorded, not verified against evidence — that's the whole point of it, but it means it can be exactly as wrong as any note in a ticket can be.

5. What changes between test and deployment? A lot. The eval set is a fixed synthetic snapshot; live traffic has a different alert mix, volume, and base rate, and the POC path has never run against a real production feed at scale. D3's retrain loop tracks a drift score — how similar recent alert text still is to what it trained on — precisely because this is expected to move, not a one-time concern. The correlation window and chain-spread constants above are hand-tuned defaults calibrated against synthetic data, not measured live traffic. And the live-Wazuh polling path is newer and far less exercised than the synthetic-replay path everything else here is tested against.

Automated mitigation (active response, Phase 14)

zuumb reads Wazuh alerts, groups them into incidents, and for each incident suggests response tasks — short instructions like "block that IP" or "lock that account". Until Phase 14 those were just notes for a person to act on by hand. Now a person can click one button and have zuumb carry the task out on the affected machine — by asking Wazuh to do it, never by running commands itself.

Three things are always true:

• Nothing happens on its own — a human clicks Approve first.
• zuumb can only do a few specific things (see the table below), never "run any command".
• Out of the box it runs in practice mode (RESPONSE_DRY_RUN=true): the button works, writes a record, and touches nothing.

Term	Plain meaning
Wazuh manager	The central Wazuh server. zuumb talks to it over HTTPS on port 55000.
Wazuh agent	A small program on a machine you want to protect. Each has a numeric agent id — 001, 006, and so on.
Active Response (AR)	A built-in Wazuh feature: the manager tells an agent to run a small, pre-installed script (block an IP, lock an account, …).
AR script	The pre-installed program the agent runs — e.g. firewall-drop (adds a firewall rule) or disable-account (locks a local user).
Dispatch	zuumb sending one "run this AR script on this agent" request to the manager.
Allowlist	The short, fixed list of actions zuumb is permitted to dispatch. Anything not on it is impossible.
Dry-run	Practice mode. Approving records what would have happened and stops.
Audit log	A table (response_actions_log) with one row per Approve click: what, where, who, when, result. Shown at /audit.
zuumb-ar	A dedicated Wazuh login zuumb uses only to dispatch AR. It can do that and nothing else.
iptables DROP rule	A Linux firewall entry that silently discards traffic from an IP. firewall-drop adds one.
RBAC	"Role-Based Access Control" — Wazuh's way of saying "this login may do X but not Y".
PowerShell / WSL	zuumb runs on Windows → PowerShell. Docker + Wazuh run under WSL (Linux). Commands below say which to use.

The safety rules, and why each one exists

Rule	Why
Only actions on a fixed allowlist	A bug or a bad LLM response can never turn into an arbitrary command.
A human clicks Approve	zuumb proposes; a person decides. No auto-fire.
Dry-run is the default	You can test the whole flow without touching a real machine.
A separate least-privilege login (zuumb-ar)	If that credential leaks it can only dispatch AR — not read alerts, delete agents, or change settings.
Every Approve is logged (dry-run and live)	There is always a record of what was, or would have been, done.
disable-user needs a second confirm	Locking an account can lock out a real person; one stray click shouldn't.
Live dispatches are rate-limited (30 s default)	A double-click or a loop can't fire a burst of real actions.

What happens when you click Approve

1.  Wazuh agent  ->  zuumb poller pulls the alert  ->  scores it  ->  groups it into an incident

2.  zuumb proposes response tasks. A task that maps to an allowlisted action gets
    TAGGED with  action + target + agent id     (e.g.  block-ip / 198.51.100.77 / 006)

3.  you click  "Approve"  on that task in the dashboard

4a.  IF  RESPONSE_DRY_RUN = true   (the default)
        -> write ONE audit row, marked "dry-run"
        -> send nothing to any host.  Done.

4b.  IF  RESPONSE_DRY_RUN = false
        -> check the 30-second rate limit (reject if a live action was just sent)
        -> POST /active-response to the Wazuh manager, logging in as zuumb-ar
        -> manager tells the agent to run  firewall-drop
        -> the agent adds a real iptables DROP rule for that IP
        -> write ONE audit row, marked "live", with the HTTP result  (e.g. ok - 200)

5.  the audit row shows at  /audit  and in the incident's "Dispatched actions"

Which file does what:

app/response/playbooks.py       proposes tasks; tags the ones that map to an action
app/response/approve.py         on Approve: dry-run log  OR  call dispatch(), then log
app/response/active_response.py  dispatch(): the ONLY code that calls the Wazuh AR API
app/db/models.py                ResponseActionLog  = the audit table
app/web/routes.py               POST /tasks/<id>/approve   and   GET /audit

The two actions zuumb can take

Action	What it does	Wazuh script	Needs from the incident	Second confirm?
block-ip	Firewall-drops an attacker's source IP on the affected host	!firewall-drop	a source IP and the agent id	no
disable-user	Locks a compromised local account on the affected host	!disable-account	a username and the agent id	yes

The ! prefix tells Wazuh "run this named script directly". Any other response task stays a manual to-do — zuumb will not dispatch it.

---

Step 1 — create the zuumb-ar login (once)

Run in WSL — this talks to the Wazuh manager. Line by line:

API='https://localhost:55000'
# log in as the manager admin (default for the 4.9.2 docker stack) and keep the token
H="Authorization: Bearer $(curl -sk -u wazuh-wui:'MyS3cr37P450r.*-' -X POST "$API/security/user/authenticate?raw=true")"
# helper: pull the numeric id out of a Wazuh JSON reply
id() { python3 -c "import sys,json;print(json.load(sys.stdin)['data']['affected_items'][0]['id'])"; }
# password: 8+ chars, upper+lower+digit+symbol, and NO '!' (bash mangles '!')
AR_PASS='Zuumb.AR.pass1'

# 1) make an empty role called "zuumb_ar"
ROLE=$(curl -sk -H "$H" -X POST "$API/security/roles" -H 'Content-Type: application/json' -d '{"name":"zuumb_ar"}' | id)
# 2) attach Wazuh's built-in policy #6 = "may run active-response commands", nothing else
curl -sk -H "$H" -X POST "$API/security/roles/$ROLE/policies?policy_ids=6" >/dev/null
# 3) create the user "zuumb-ar"
USER=$(curl -sk -H "$H" -X POST "$API/security/users" -H 'Content-Type: application/json' -d "{\"username\":\"zuumb-ar\",\"password\":\"$AR_PASS\"}" | id)
# 4) give the user the role
curl -sk -H "$H" -X POST "$API/security/users/$USER/roles?role_ids=$ROLE" >/dev/null

# --- prove the login is correctly limited ---
ART=$(curl -sk -u "zuumb-ar:$AR_PASS" -X POST "$API/security/user/authenticate?raw=true"); sleep 2
curl -sk -o /dev/null -w 'dispatch AR  -> %{http_code}  (want 200 = allowed)\n' -H "Authorization: Bearer $ART" \
  -X PUT "$API/active-response?agents_list=001" -H 'Content-Type: application/json' -d '{"command":"!firewall-drop","arguments":["203.0.113.9"]}'
curl -sk -o /dev/null -w 'delete agent -> %{http_code}  (want 403 = blocked)\n' -H "Authorization: Bearer $ART" \
  -X DELETE "$API/agents?agents_list=999&status=all&older_than=0s"

200 then 403 = success: zuumb-ar may dispatch active response and nothing else.

Step 2 — tell zuumb about it

In .env (Windows, repo root):

WAZUH_AR_API_URL=https://localhost:55000
WAZUH_AR_API_USER=zuumb-ar
WAZUH_AR_API_PASSWORD=Zuumb.AR.pass1
RESPONSE_DRY_RUN=true            # leave true until you deliberately go live
RESPONSE_RATE_LIMIT_SECONDS=30

Step 3 — watch it work, safely (dry-run)

PowerShell = zuumb commands, run from D:\ai-soc-xdr with .venv\Scripts\python.exe. WSL = the docker exec ... checks. A zuumb command in WSL fails with No module named 'scripts'.

3a. start from an empty database — PowerShell, uvicorn stopped:

Rename-Item zuumb.db zuumb.db.bak     # keeps your live data aside; you restore it in step 5

Why this matters: on a database that already has live data, the demo alerts get absorbed into an existing host incident and older proposed tasks (same title) hide the freshly tagged ones — so no ⚡ action tag appears and there is nothing to approve. A fresh DB is required for the walkthrough, not just tidier.

3b. load the sample alerts — PowerShell:

.venv\Scripts\python.exe -m scripts.run_poc --offline

What it does: reads the sample alert files in data\synthetic_alerts\, scores them (--offline = a keyword scorer, no paid LLM), groups them into incidents, and proposes response tasks. data\synthetic_alerts\ar_demo.jsonl is built to produce one block-ip task and one disable-user task on agent-lab-01.

3c. list the tasks zuumb tagged as dispatchable — PowerShell:

.venv\Scripts\python.exe -c "from app.db.session import get_session; from app.db.models import Task; from sqlmodel import select; s=get_session(); [print(f'incident {t.incident_id}  task {t.id}  ->  {t.action}  {t.action_target}  on agent {t.agent_id}') for t in s.exec(select(Task).where(Task.action!=None)).all()]"

What it does: prints every one-click action task. Note the task <N> number of a block-ip line whose agent id is a real agent (list them in WSL with docker exec wazuh49-wazuh.manager-1 /var/ossec/bin/agent_control -l).

3d. start zuumb — PowerShell:

.venv\Scripts\python.exe -m uvicorn app.main:app --port 8000

Open http://localhost:8000.

3e. approve — in the browser:

1. Incidents list → click the incident.
2. Scroll to Response tasks. The dispatchable one shows a tag like ⚡ block-ip → 198.51.100.77 · agent 006 and a yellow DRY-RUN badge.
3. Click Approve.
   • block-ip → done at once.
   • disable-user → the button turns red and says Confirm dispatch; click it again to go through (this is the "second confirm" safety rule).

What just happened: RESPONSE_DRY_RUN=true, so zuumb wrote an audit row saying "would have run block-ip on agent 006" and stopped. Nothing was sent anywhere.

3f. check the record — in the browser:

Look at	You should see
top nav → Audit (/audit)	one row, MODE = dry-run, RESULT = ok
the incident page → Dispatched actions	the same row

Step 4 — do it for real (live)

⚠️ This runs a real firewall command on the target agent. Use a throwaway agent (agent-lab-01 / agent-lab-02 from docker-compose.agents.yml), never a machine you care about.

4a. flip the switch — PowerShell (stop uvicorn with Ctrl+C first):

(Get-Content .env) -replace 'RESPONSE_DRY_RUN=true','RESPONSE_DRY_RUN=false' | Set-Content .env
.venv\Scripts\python.exe -m uvicorn app.main:app --port 8000

What it does: RESPONSE_DRY_RUN=false means Approve now really dispatches.

4b. approve the same block-ip task again — browser, or PowerShell:

curl.exe -s -X POST http://localhost:8000/tasks/<TASK-NUMBER>/approve

Put the number from step 3c where <TASK-NUMBER> is (e.g. 194). Check in the UI: /audit — the newest row now says MODE = live, RESULT = ok · 200.

4c. confirm it actually happened — WSL (use your task's target IP):

docker exec zuumb-agents-agent-lab-01-1 iptables -S | grep 198.51.100.77
docker exec zuumb-agents-agent-lab-01-1 tail -3 /var/ossec/logs/active-responses.log

Expected:

-A INPUT   -s 198.51.100.77/32 -j DROP        <- the firewall now blocks that IP
-A FORWARD -s 198.51.100.77/32 -j DROP
...
active-response/bin/firewall-drop: ... "extra_args":["198.51.100.77"] ...
active-response/bin/firewall-drop: Ended

The -j DROP line is the whole point: on your click, zuumb had Wazuh block a real IP on a real host.

Step 5 — put everything back

5a. back to practice mode — PowerShell (Ctrl+C uvicorn):

(Get-Content .env) -replace 'RESPONSE_DRY_RUN=false','RESPONSE_DRY_RUN=true' | Set-Content .env

5b. remove the firewall rule — WSL (your target IP):

docker exec zuumb-agents-agent-lab-01-1 sh -c \
  'iptables -D INPUT -s 198.51.100.77/32 -j DROP; iptables -D FORWARD -s 198.51.100.77/32 -j DROP; \
   iptables -S | grep 198.51.100.77 || echo cleared'

Or docker compose -f docker-compose.agents.yml restart agent-lab-01 — the container's firewall rules aren't saved, so a restart wipes them.

5c. restore your real database (if you renamed it in 3a) — PowerShell:

Remove-Item zuumb.db; Rename-Item zuumb.db.bak zuumb.db

5d. restart zuumb — PowerShell:

.venv\Scripts\python.exe -m uvicorn app.main:app --port 8000

It's back in dry-run. The /audit rows stay — that history is intentional.

Publishing a release (maintainers)

Images are built and pushed by .github/workflows/publish.yml — no secrets to set, it uses the repo's automatic GITHUB_TOKEN.

git tag v0.1.0
git push origin v0.1.0

The workflow builds linux/amd64 + linux/arm64 and pushes ghcr.io/nucleiav/zuumb tagged 0.1.0 and latest.

One manual step after the first publish: GHCR packages default to private even on a public repo. Go to the repo → Packages → zuumb → Package settings → Change visibility → Public. Skip this and docker compose up fails for everyone but you.

The Dockerfile is multi-stage (deps in a build stage, only the venv + app/, prompts/, scripts/, eval/, data/synthetic_alerts/ in the runtime layer), runs as a non-root user, and carries no dev tooling or bind mounts. .dockerignore keeps .env, *.db, the landing page, and the planning docs out of the image.

Build status

• Phase 1 scaffold + infra — done (Wazuh external; infra is Postgres-only).
• Phase 2 ingestion (synthetic alert JSON → DB) — done.
• Phase 3 triage agent (Claude → structured verdict) — done.
• Phase 4 POC checkpoint (scripts/run_poc.py) — done, demoed with real API.
• Phase 5 correlation engine (app/correlation/engine.py) — done.
• Phase 6 dashboard v1 (FastAPI + Jinja2) — done.
• Phase 6b dashboard viz upgrade (brand theme toggle + Chart.js charts) — done.
• Phase 6c Wazuh-parity UI pass (breadcrumb nav, widget chrome, dense grid, severity palette, tight tables) — done.
• Phase 7 attack chain stitcher (app/attack_chain/stitcher.py) + /chains view — done.
• Phase 8 response layer (app/response/playbooks.py, propose-only, no execution) — done.
• Phase 9 feedback loop (app/feedback/logger.py, analyst override → few-shot in triage prompt) — done.
• Phase 10 eval harness (eval/run_eval.py) — done; current run against the 104-alert labeled set: baseline acc 0.894, malicious recall 0.70, 0 malicious ever scored benign (eval/RESULTS.md, which also keeps the original 34-alert run for the record). Directional only: self-labeled synthetic set, one annotator, default (non-zero) model temperature — not a certified benchmark.
• Phase 12 live Wazuh ingestion (app/ingestion/wazuh_client.py poller + app/pipeline.py) — done against a live 4.9.2 stack.
• Phase 13 attack chains at scale — CHAIN_MAX_ENTITY_SPREAD knob, scripts/chain_quality.py diagnostic, container agent lab (docker/agent/, docker-compose.agents.yml); validation ongoing against real traffic.
• Phase 14 human-approved active response — allowlisted Wazuh AR dispatch (app/response/active_response.py), approve→dispatch flow with dry-run default + audit log (app/response/approve.py, /audit); verified live against a throwaway agent (real iptables DROP rule).
• Phase 15 public Docker distribution — multi-stage Dockerfile, .github/workflows/publish.yml (multi-arch → GHCR on v*.*.*), docker-compose.yml (pull) + docker-compose.dev.yml (build).

Detection track — pluggable engines alongside Wazuh's own rule alerts, see What's rule-based, what's learned, and what's a language model above for how these fit together:

• D1 auth-log anomaly detector (detectors/authlog/, deterministic ML, no LLM) — done, runs continuously as its own container against the live Wazuh alerts index.
• D2 network beacon detector (detectors/netflow/, deterministic, no ML/LLM) — done, CLI only; no live flow-data source exists in this lab to schedule it against.
• D3 second-opinion classifier (app/triage/second_opinion.py, deterministic ML, no LLM) — done, advisory cross-check on the LLM's primary verdict.
• D4 retrain loop for D3 (app/triage/retrain.py) — done, weekly, gated so a retrain can never make the live model worse.
• D5 adversarial robustness check on D3 (eval/adversarial.py) — done; findings in eval/ADVERSARIAL.md.
• Incident similarity (app/correlation/similarity.py, local sentence embedding, deterministic given a fixed model, no LLM) — done, advisory-only signal alongside the deterministic entity/time correlation, never replaces it.
• Manual incident linking (app/correlation/manual_link.py) — done, a human-confirmed record separate from correlation, similarity, and attack chains; drives nothing downstream (guarded by a test asserting stitcher.py/engine.py never reference it).