Skip to content

A Kali Purple analyst workstation (garuda) on Saruman, not a second SOC #921

Closed

Description

@Gerrrt

A Kali Purple guest on Saruman: a defender's analyst workstation, not a second SOC.

Where, and why there

Saruman, on ImaginationLAN. ADR-0007 splits the lab as "Saruman defends, ifrit attacks", and a defender's machine belongs on the defended side, beside the SOC on odin. It does not go on ifrit: the range is built so that its targets have no route (ADR-0014), and defensive tooling sitting inside the attack range would blur that split.

Proposed name: garuda, a free summon (unused in docs/, stacks/, tofu/, ansible/ and packer/ as of 2026-10-06). The address comes from provisioning, not from this issue.

What it is for, and what it is not

Kali Purple bundles a whole SOC. Most of that already runs here and is not duplicated:

Purple bundles Already here
Elastic SIEM Wazuh on odin (ADR-0030)
Zeek Zeek on fenrir, from the lab-bridge mirror (ADR-0068)
Suricata Suricata on morpheus
Response tooling Velociraptor on odin

A second SIEM would split the telemetry and the RAM between two half-fed systems. Its SIEM and sensors stay off. What it adds:

  • An analyst desktop: CyberChef, Wireshark, and the tools for working an alert that Wazuh raised.
  • Vulnerability scanning of the lab (OpenVAS/GVM), pointed at the lab domain's guests and nothing outside VLAN 30.
  • Case management (TheHive), if it earns its keep.
  • The purple half of an exercise: attack from ifrit's Kali (Build tpl-kali (VMID 902) on ifrit once it exists #790), then watch and respond from here.
  • The home of dotfiles-Defense. It is the defensive distro, so the Defense role layer lands here, mirroring Offense on ifrit's Kali. One on-demand VM per dotfiles OS layer on Saruman #920 is updated to say so.

Sizing and lifecycle

About 4 vCPU, 8 GB RAM, 80 GB thin on large_data (SSD). Saruman had 54.5 GiB of RAM free on 2026-10-06. Always on, like the rest of the domain, and decided 2026-10-06: it is not tagged on-demand, unlike #920's dotfiles VMs. So HypervisorGuestStopped covers it, and it counts against Saruman's RAM continuously (54.5 GiB was free on 2026-10-06). It is built like the rest of the lab, with a Packer template from phoenix (ADR-0074) and an OpenTofu guest (ADR-0076).

Open before building

  • The OpenVAS scans' scope has to be written down (VLAN 30 only), and the scans will trip Suricata and Wazuh. That is useful as a detection test, but it needs a silence owned by this issue while a scan runs.
  • Does it join the lab domain? Decided 2026-10-06: no. It stays standalone beside the domain, like odin and Run BloodHound CE where nothing attacks it #451's BloodHound. The reasoning is in the comments.

Activity

  1. added this to the Tier extras milestone on Oct 6, 2026
  2. added
    enhancementNew feature or request
    labstacks/lab on alexander (VLAN 30)
    socstacks/soc on odin (VLAN 30)
    on Oct 6, 2026
  3. Gerrrt commented on Oct 6, 2026

    @Gerrrt
    OwnerAuthor

    2026-10-06: garuda is always on, not on-demand. It runs continuously, like the domain's other guests. Only #920's dotfiles VMs stay on-demand. The body is corrected.

    Being a standing member of the estate means more than staying powered on:

    • Alerting: HypervisorGuestStopped covers it, because it is not tagged on-demand, so a stopped garuda is a finding.
    • Telemetry: the same agents as the domain's other guests, with liveness and telemetry crossing per ADR-0028, and enrolled in Wazuh and Velociraptor on odin. The SOC watches its own analyst's machine.
    • Patching: Kali is rolling, so it updates on the same cadence as the other guests, and the patch-state collector should report it.
    • Backups: it is covered like the other standing guests. It holds case notes and scan results that are not in git.
    • The domain question is now answered: "like the rest of the domain" reads as a domain member. That is the assumption for the build, unless that wasn't meant.
  4. Gerrrt commented on Oct 6, 2026

    @Gerrrt
    OwnerAuthor

    2026-10-06: correction. garuda is not a domain member. It is always on beside the domain, not in it. This replaces the assumption in the previous comment.

    • The domain is the target. Give the lab domain users and deliberate weaknesses, one tag at a time #449 gives it deliberate weaknesses, and the exercises from ifrit aim to take it over. A domain-joined analyst workstation would fall with it: Domain Admin over the domain would mean admin over garuda, its case notes and its scan results. This is the separation real SOCs keep, with analyst machines on a tier the defended domain cannot administer.
    • The domain is reverted. A member's secure channel breaks across reverts, so garuda would keep dropping out of the domain.
    • Precedent. Run BloodHound CE where nothing attacks it #451 places BloodHound "not on the domain itself. It analyses the domain; it should survive the domain being reverted." odin is standalone for the same reason.
    • Nothing it does needs membership. OpenVAS can run authenticated scans with a domain credential, and LDAP queries work from a non-member.

    Practising a domain-joined analyst workflow, if that's ever wanted, belongs on a separate domain-joined desktop guest, not on the machine that holds the evidence.

    The other "always on" consequences in the previous comment stand: alerting, telemetry, Wazuh and Velociraptor enrolment, patching and backups.

  5. Gerrrt commented on Oct 10, 2026

    @Gerrrt
    OwnerAuthor

    2026-10-10: phase 1 built. garuda is up at 10.0.30.62 (VMID 162), with Purple's desktop and tools and the dotfiles-Defense layer. Authored in #1116; the full as-run log is in build-the-analyst-workstation.md §11.

    Template 910 (tpl-kali-saruman, ADR-0091)

    • Built twice from phoenix: 26m25s, then a -force rebuild in 26m11s.
    • packer-smoke.sh 910 passed after each.

    The guest

    • A targeted tofu apply against the shared state: 2 added, 0 changed, 0 destroyed (the analyst pool and garuda).
    • Tagged analyst, not on-demand, so HypervisorGuestStopped covers it.
    • onboot: 1, Secure Boot keys off, 4 vCPU / 8 GiB.

    Purple, without its SOC

    • kali-desktop-xfce, kali-themes-purple, CyberChef 11.3.0, Wireshark/tshark 4.6.6 (non-root capture via the wireshark group), Suricata 8.0.7, YARA, jq, and Docker 28.5.2 with Compose 2.40.3.
    • suricata.service is disabled, and no Suricata, Zeek, Elastic, Kibana or Wazuh listener is up.
    • No Zeek: Kali's zeek 5.1.1 needs libc6 (< 2.38) and cannot be installed on current Rolling. fenrir is the sensor; captures can be run through the pinned zeek/zeek image.

    dotfiles, in ~/code/dotgibson/

    • dotfiles-Debian v0.1.59: bootstrap.sh exit 0, using the Kali tier.
    • dotfiles-Defense v1.0.132: bootstrap.sh exit 0, and its probe confirms docker compose is available, so siemup would work. siemup is down, as ADR-0091 requires.
      • 7 optional forensics tools missing: zeek chainsaw hayabusa sigma-cli velociraptor vol log2timeline.py.
    • core doctor (dotfiles-core 7.14.0): exit 0, nothing expected missing.

    Found on the way, filed separately

    Still open in phase 1

    • The lab ingest token and stacks/analyst (runbook §6).
    • Patch-state and the golem backup job (§8).
    • The telemetry and desktop checks (§9).
    • Garrett: set the console password with ssh -t analyst@10.0.30.62 sudo passwd analyst.

    Phases 2 (Wazuh/Velociraptor enrolment) and 3 (OpenVAS) are unchanged.

  6. Gerrrt commented on Oct 10, 2026

    @Gerrrt
    OwnerAuthor

    2026-10-10: phase 1 is built and reporting. garuda now sends telemetry to the lab, is backed up, and passed runbook §9 apart from the desktop line. The as-run log is in §11 (#1149).

    §6, telemetry (#1144)

    • garuda has its own age key, and its ingest token is set on alexander and garuda. The token arrived on stdin only, so it was never an argument, and the two hashes match.
    • alexander's lab stack was pulled to main and brought up: all healthy, no 401s.
    • stacks/analyst is up on garuda: Alloy and its socket proxy, both healthy.

    §8, collectors and backup

    • patch-state is installed, showing homelab_apt_upgrades_pending{host="garuda"} 0.
    • 162 joined golem-nightly. A test snapshot backup ran in 9m03s, encrypted, and the guest's filesystem freeze and thaw were clean.
      • That job held only odin (160) beforehand. The domain's six were not in it, and I left that as it was.

    §9, checks

    • Estate: homelab_guest_running 1 and homelab_guest_on_demand 0, so HypervisorGuestStopped covers it.
    • Lab: up = 1 for garuda-alloy and garuda-metrics, and garuda's journal is in Loki.
    • No second SOC: only analyst-alloy and analyst-docker-socket-proxy run. SSH (22) is the only port open beyond loopback.
    • dotfiles: core doctor exits 0.
    • Reboot: qm reboot 162 brought everything above back with nothing typed (agent back in 41s).

    Earlier the same day, #1128 was fixed in place on garuda: / is now 79 GB, with a 4 GB swapfile.

    Left for Garrett

    • Set the console password: ssh -t analyst@10.0.30.62 sudo passwd analyst, typed at the prompt.
    • Run §9's desktop line from Hicks: the Purple XFCE session, Wireshark opening a capture, CyberChef.
    • Watch the greeter while doing it. lightdm's account ships expired (1970-01-02), so user@968.service fails on every boot and systemd reads degraded. lightdm itself is running, and the desktop check will show whether that matters.

    Still open on #921: phase 2 (Linux Wazuh and Velociraptor enrolment), phase 3 (OpenVAS), #1127 (the first-boot cloud-init race) and #1128's template-side fix.

  7. Gerrrt commented on Oct 10, 2026

    @Gerrrt
    OwnerAuthor

    2026-10-10: phase 2 done. garuda is enrolled in odin's Wazuh and Velociraptor, the first Linux guest in either. The decision is ADR-0092, the change #1163, and the as-run is in #1164 (runbook §11).

    • Packages. Two Debian packages are staged on odin beside the six's MSIs, and 8448 now answers .62.
      • Wazuh agent 4.14.8: the manager's version, pinned by sha256. The hash is backed by Wazuh's signed apt InRelease.
      • Velociraptor client 0.77.3: built by the server with its config inside. garuda checked its hash against the one read from odin over SSH.
    • Wazuh. Agent 009 garuda, Active.
      • The enrolment password went in on stdin only, and its hash matched odin's authd.pass.
      • authd.pass was removed afterwards.
    • Velociraptor. velociraptor_client is active with a connection to odin:8000.
      • The server counts 7 client connections and one enrolment, garuda's.
      • The downloaded packages were deleted from garuda, because the .deb carries the enrolment nonce.
    • Alerting. WazuhAgentsNotConnected now expects garuda's agent while garuda's Alloy is up.
      • It is deployed on alexander and inactive.
      • It read pending for a few minutes first, because the collector ran six seconds after the agent connected.

    Left for Garrett:

    • In the Velociraptor GUI from Hicks, check that a client with hostname garuda is listed. A clients() query run from a second process on the server returned nothing, so I couldn't confirm it from here.
    • The desktop check and the console password, from phase 1.

    Next: phase 3, OpenVAS.

  8. Gerrrt commented on Oct 11, 2026

    @Gerrrt
    OwnerAuthor

    All three phases are built, proven and merged. garuda is VMID 162 at 10.0.30.62 on Saruman: Kali Purple as a defender's analyst workstation, not a second SOC.

    Phase 1 — the workstation (#1116, as-run #1149; in-place #1128 fix #1135)

    • Template 910 (tpl-kali-saruman), the guest in tofu's analyst pool, always on, not domain-joined.
    • Purple's desktop and tools with the SIEM/sensor daemons off; dotfiles-Debian + dotfiles-Defense installed.
    • Reports to the lab (ingest token, stacks/analyst), patch-state, and backed up by golem.

    Phase 2 — SOC enrolment (#1163, as-run #1164, ADR-0092)

    • The first Linux guest in odin's Wazuh and Velociraptor, from Debian packages staged on odin's 8448.
    • Wazuh agent 009 Active; Velociraptor client connected. WazuhAgentsNotConnected expects garuda.

    Phase 3 — OpenVAS (#1166, ADR-0093)

    • Kali's gvm packages, run on demand, scoped to the domain's six.
    • Fenced at layer 2: the scanner runs in a network namespace whose only route to the segment is a macvlan (.63) behind an nftables netdev egress filter dropping every frame — IP, ARP, raw — to anything but the six. Proved: out-of-scope arping/TCP/ICMP dropped, in-scope reachable.

    Decisions recorded along the way: not a second SIEM (ADR-0091), standalone not domain-joined, always-on, template 910 vs 902, packages staged on odin (ADR-0092), and the on-demand layer-2-fenced scanner (ADR-0093).

    Left to Garrett (hands-on-console / UI, can't be done from here), tracked on their own:

    • the desktop check from Hicks and the console password;
    • confirm the garuda client in the Velociraptor GUI;
    • change GVM's admin password in the web UI and delete /root/gvm-install.log.

    Follow-ups spun out and closed separately: #1127 (Kali first-boot cloud-init), #1128 (template root growth). #1164 (phase-2 as-run) lands via auto-merge.

  9. Gerrrt commented on Oct 14, 2026

    @Gerrrt
    OwnerAuthor

    §9 degraded state resolved on garuda (162).

    The one failed unit left after the build — user@968.service ("User Manager for UID 968") — was lightdm failing PAM because its shadow account-expiry was pinned to the epoch (1970-01-02):

    pam_unix(systemd-user:account): account lightdm has expired (account expired)
    user@968.service: Failed at step PAM ... status=224/PAM
    

    It's not lightdm-specific: ~14 Kali service accounts ship expired at the epoch (lightdm _gvm inetsim mysql tcpdump Debian-snmp pcscd rtkit dnsmasq nm-openvpn nm-openconnect systemd-network systemd-timesync _dhcpcd). lightdm is just the only one that surfaces, because it's the only expired account systemd starts a user@<uid>.service for (it lingers a session for the greeter); the rest are pure daemons that never hit PAM's account path. root, analyst, sshd, messagebus are clean. The greeter itself was always fine — only the systemd state was dirty.

    Fixed live on garuda — cleared the bogus expiry on every UID<1000 account that had it, then started the unit:

    chage -E -1 <account>   # for each UID<1000 account with shadow field-8 == 1
    systemctl start user@968.service

    Now: account expires: never, user@968 active, 0 failed units, systemctl is-system-running → running.

    It's a template defect (910, and the shared 902/903 built from the same packer/kali.pkr.hcl + preseed), coming from Kali's install/postinst layer — our build touches no account expiry. Tracked for a proper fix at the template in #1194, so fresh clones aren't born degraded.

    That clears the §9 degraded line. Remaining #921 console/UI items for you: §9 desktop visual check from Hicks, console password, Velociraptor GUI client check, and the GVM admin password change in the UI + deleting /root/gvm-install.log (still present).

  10. Gerrrt commented on Oct 14, 2026

    @Gerrrt
    OwnerAuthor

    Template fix for the §9 degraded state is done and verified.

    The epoch account-expiry that made clones boot degraded (lightdm's user@<uid>.service failing PAM) is now fixed in the Kali preseed itself, not just live on garuda:

    So every future clone of 910 comes up clean. garuda (162) itself was already fixed live.

    Sibling template 902 (ifrit's Kali, same preseed) carries the same defect until it's rebuilt; tracked in #1196 (tied to #790).

    Remaining #921 console/UI items for Garrett are unchanged: §9 desktop visual check from Hicks, console password, Velociraptor GUI client check, and the GVM admin-password change in the UI + deleting /root/gvm-install.log.

  11. Gerrrt commented on Oct 14, 2026

    @Gerrrt
    OwnerAuthor

    The §11 as-run write-up of the degraded/epoch-expiry fix is merged — PR #1197 (squash 6b0d72a). It records the root cause, the live fix on garuda, the preseed late_command template fix, and the 910 rebuild + smoke + clean-boot clone verification, scoped so the guarantee applies to a rebuilt 910 (902 still owed, #1196).

    That closes out the degraded item in docs as well as in code. Remaining #921 console/UI items for you are unchanged: §9 desktop visual check from Hicks, console password, Velociraptor GUI client check, and the GVM admin-password change in the UI + deleting /root/gvm-install.log.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestlabstacks/lab on alexander (VLAN 30)socstacks/soc on odin (VLAN 30)

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions