Skip to content
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,7 @@ documents for different readers.
at all. Every pinned digest is also scanned weekly for fixable HIGH and
CRITICAL CVEs, and each image with findings has an open issue until a scan
finds it clean.
- **Documented decisions and runbooks.** 95 ADRs covering what was chosen
- **Documented decisions and runbooks.** 96 ADRs covering what was chosen
and what was rejected — including the costs accepted knowingly; 49
runbooks for the operations that are easy to get wrong at 1am, one of which
is the handover page a successor reads first. Every critical alert links to
Expand Down Expand Up @@ -224,7 +224,7 @@ inbound passes may still reach it. Data flow and the maintained Mermaid topology
│ ├── architecture.md network.md hardware.md
│ ├── observability.md security.md roadmap.md changelog.md
│ ├── diagrams/ # the network diagram (SVG) and its predecessors
│ ├── adr/ # 95 ADRs — architecture decision records
│ ├── adr/ # 96 ADRs — architecture decision records
│ ├── exercises/ # purple-team exercise records — see its README
│ └── runbooks/ # 49 runbooks; successor-handover.md is the front door
└── Makefile # make help
Expand Down Expand Up @@ -399,7 +399,7 @@ is [`docs/changelog.md`](docs/changelog.md), dated and never rewritten.

**Every purchase still outstanding is in one place**, the roadmap's
[*Everything still to buy*](docs/roadmap.md#everything-still-to-buy):
11 items now, 0 later, and a rule that nothing joins them without a
10 items now, 0 later, and a rule that nothing joins them without a
decision. This sentence used to carry the list itself, name three purchases
coupled to the UPS work and omit the tier's host entirely, which is how one
ProDesk came to be bought for two jobs. It names no current item for the
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,15 @@

**Status:** Accepted · 2026-09

> [!NOTE]
> **The *Disk* row of the decision table is superseded. The text below is
> left as written, per ADR-0001.**
> [ADR-0096](0096-fit-ifrits-range-disk-as-a-sata-ssd-in-the-tinys-bay.md)
> puts the range disk in the M80q's 2.5" bay as a SATA SSD, decided
> 2026-10-16, and keeps the 256 GB M.2 as the boot disk. The row's premise,
> flash rather than spindles for snapshot revert, stands; what changed is
> that the box has a bay as well as the slot.
>
> [!NOTE]
> The ADR-0007 sentence this ADR quotes under "The fleet's measured
> bottleneck is spindles, not RAM" was measured on 2026-09-20
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
# ADR-0096: Fit ifrit's range disk as a SATA SSD in the Tiny's bay

**Status:** Accepted · 2026-10 · supersedes the *Disk* row of
[ADR-0017](0017-buy-ifrit-for-iops-and-keep-the-range-disposable.md)

## Context

[ADR-0017](0017-buy-ifrit-for-iops-and-keep-the-range-disposable.md) decided
`ifrit`'s disk as "**NVMe.** Not a spindle, and not a mirror", on the ground
that the range's core operation is snapshot revert and "a chassis without an
M.2 slot cannot be given one". The box that was bought on 2026-10-05, a
first-generation ThinkCentre M80q, has that slot, and it is occupied by the
256 GB M.2 the refurbisher fitted ([`hardware.md`](../hardware.md)). The
roadmap's purchase row therefore read "1 TB NVMe SSD", and the move runbook's
§1.3 had it *replace* the 256 GB drive, "there is nowhere to fit both".

That last sentence was wrong. Lenovo's PSREF sheet for the first-generation
M80q gives its storage as "up to two drives, 1x 2.5" HDD + 1x M.2 SSD", the
2.5" bay on SATA 6 Gb/s. The bay takes a 2.5" SSD as readily as a disk, and
it is empty on a Tiny sold with an M.2 only. The tray and its flat SATA
cable are in this chassis, checked on 2026-10-16; a refurbished box can ship
without them, in which case Lenovo sells them as the Tiny tray kit
`5M10U50268` with the `00XL211` cable.

The operator chose on 2026-10-16 to put the range disk in that bay as a SATA
SSD rather than swap the M.2.

What ADR-0017 was actually guarding against is worth restating, because the
choice turns on it. Its argument was flash against spindles: the range reverts
snapshots all day, and the fleet's measured bottleneck was a 7.2K mirror doing
741 random write IOPS at queue depth 1 against 7,952 from a SATA SSD pair, as
its own note records. A SATA SSD is on the right side of that line by an order
of magnitude. What SATA gives up against NVMe is the next order: sequential
throughput capped near 550 MB/s against several GB/s, and random IOPS in the
tens of thousands rather than the hundreds. For four or five guests on one
host, reverting one at a time, that is not the bottleneck; the range's
activity is attack tooling, not storage benchmarking.

## Decision

**`ifrit`'s range disk is a 1 TB 2.5" SATA SSD in the Tiny's drive bay. The
256 GB M.2 stays, as the hypervisor's boot disk.**

- **The bay, not the slot.** The SATA SSD mounts in the Tiny's 2.5" tray and
connects by the flat SATA cable to the board, as the M80q hardware
maintenance manual shows. Both are in the chassis, so no kit is bought.
- **Two drives, two jobs.** The 256 GB M.2 is the Proxmox boot and root disk;
the 1 TB SATA SSD is the guest storage, where templates, the attack VM and
the targets' disks live and are reverted. The boot disk is never part of
the range's churn, which is a cleaner layout than ADR-0017's single drive
and is the one real gain of this change.
- **Nothing else in ADR-0017 moves.** Socketed RAM to 32 GB, the quiet
form, one NIC, and the range being maintained by nobody all stand.

## Consequences

- `docs/roadmap.md`'s row becomes a 2.5" SATA SSD;
[`hardware.md`](../hardware.md) and the two runbooks that fit and
then use the drive say the same. The move runbook's §1.3 no longer swaps
the M.2 and no longer keeps a 256 GB drive "with the spare parts".
- The range's storage ceiling is SATA's. If a future exercise is ever
storage-bound, the M.2 slot is still there, and the fix is a larger NVMe
in it with the roles swapped, not a new chassis. That is the condition that
would reopen this decision.
- The 2.5" bay's cable is a flat flexible one that routes under the board on
a Tiny; fitting it is a case-open job done once, in the move runbook's
§1.3, before the box is racked.

## Alternatives considered

- **A 1 TB NVMe replacing the 256 GB M.2**, as the runbook had it. One drive
for boot and guests, the 256 GB left on a shelf, and the fastest option.
Rejected by the operator on 2026-10-16 in favour of SATA; the layout above
is also better separated.
- **A 1 TB NVMe plus the 256 GB M.2 moved to the bay** via an M.2 SATA
drive. Not possible: the 256 GB drive is NVMe, and the bay is SATA.
- **Leaving the 256 GB M.2 as the only disk.** ADR-0017 sized the range at
four or five guests with templates; 256 GB is below that before the first
snapshot.
25 changes: 25 additions & 0 deletions docs/changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,31 @@ roadmap as it read that day, and the *Done* entries keep the shape they had
there. `check_docs.py` does not check this file, for the reason its module
docstring gives: it is a record, not a claim about now.

## 2026-10-16

- **`mjolnir` gets 2U four-post rails, and two bought rows leave the table**
([#919](https://github.com/Gerrrt/HomeLab/issues/919)). The rails the UPS
sits on now sag under it; `hardware.md` had already said it needs rear
support. A 2U four-post pair rated above its weight joins *Everything
still to buy*, Rack Solutions 2UKIT-109 named for its 10–31.75" range,
75 lb rating and 18-gauge steel, and the move runbook racks the UPS on
them and leaves the old rails out. The short patch leads in the VLAN
colours and the spare M6 cage nuts leave the table: the operator already
has both. Eleven rows become ten.

- **`ifrit`'s range disk is a SATA SSD in the Tiny's bay, not an NVMe swap**
([ADR-0096](adr/0096-fit-ifrits-range-disk-as-a-sata-ssd-in-the-tinys-bay.md),
[#421](https://github.com/Gerrrt/HomeLab/issues/421)). The move runbook had
the 1 TB drive replace the 256 GB M.2 because "there is nowhere to fit
both". Lenovo's PSREF sheet says otherwise: the first-generation M80q takes
a 2.5" SATA drive as well as one M.2. The operator chose SATA, so the
256 GB M.2 stays as the boot disk and the 1 TB SSD carries the guests.
ADR-0017's *Disk* row is superseded and carries the note; its premise,
flash over spindles, stands. The roadmap row becomes a 2.5" SATA SSD; the
bay's tray and cable are already in the chassis, checked the same day, so
no kit joins the table. The runbook's §1.3 fits the drive in the bay
instead of swapping the M.2.

## 2026-10-15

- **Both front-mounted trays wobble, so they get four-post support rails**
Expand Down
17 changes: 13 additions & 4 deletions docs/hardware.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,9 @@ the table below.
**It is set to 29" post to post**, the old 9U frame's depth, measured
2026-10-12. The DL360's rails and the TP-Link's shelf already fit that
spacing. The DL360 Gen9 is about 70 cm (27.5") deep, the deepest thing here,
and its rails need four posts. The UPS needs rear support. Open frame suits a
and its rails need four posts. The UPS needs rear support, and the rails it
sits on now sag under it, so a 2U four-post pair rated above its weight is
to buy, decided 2026-10-16. Open frame suits a
fanless switch. Any four-post shelf bought new must fit a 29"-deep rack,
which its listing states, and bolt to both posts without forcing.

Expand Down Expand Up @@ -126,7 +128,7 @@ CRS326 with them.
| U5–U9 | `smaug`, the TS150, on its side on a four-post shelf | Heavy, so low. Off its long cord: on the PDU directly |
| U4 | 10-outlet PDU | Fed by the UPS, close to it, so the cords are short |
| U3 | HPE ProLiant DL360 Gen9 (`shiva`/`Saruman`) | Heavy and deep, so low |
| U1–U2 | APC Smart-UPS X 1500 (`mjolnir`) | Heaviest, at the bottom |
| U1–U2 | APC Smart-UPS X 1500 (`mjolnir`) | Heaviest, at the bottom, on new 2U four-post rails: the ones it has sag, decided 2026-10-16 |

Twenty-nine units are used and three are left as gaps (U11, U13, U17), so
there is no headroom for another server. If the rack runs out, the 1U
Expand Down Expand Up @@ -206,10 +208,17 @@ down so it cannot slide off or hang by its cable.
**`ifrit` was bought on 2026-10-05:** eBay item 800466163205, "Lenovo
ThinkCentre M80q Micro i7-10700T 2GHz 8GB 256GB SSD", refurbished. It is a
1-litre Tiny chassis, so 1U. It is not usable as the range yet. ADR-0017
asks for 32 GB of socketed RAM and NVMe, and two parts are still to buy:
asks for 32 GB of socketed RAM and flash storage, and
[ADR-0096](adr/0096-fit-ifrits-range-disk-as-a-sata-ssd-in-the-tinys-bay.md)
puts the range disk in the chassis's 2.5" bay as a SATA SSD, decided
2026-10-16, with the 256 GB M.2 it shipped with kept as the boot disk. Still
to buy:

- 2 × 16 GB DDR4 SO-DIMM, replacing the 8 GB it ships with;
- a 1 TB NVMe SSD, for the range's snapshot reverts.
- a 1 TB 2.5" SATA SSD, for the range's guests and their snapshot reverts.

The bay's tray and its flat SATA cable are in the chassis, checked
2026-10-16, so no kit is needed.

**A VGA card for optional port 1 was ordered on 2026-10-14** from Amazon:
Lenovo FRU `01AJ935` (also sold as `5C50W00901`). It replaces the
Expand Down
15 changes: 8 additions & 7 deletions docs/roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -274,7 +274,7 @@ Gated on the domain, on a household observation, or on something to publish.
- **[#421](https://github.com/Gerrrt/HomeLab/issues/421) Buy `ifrit` and build
the range.** Gated on #414 and the SOC: an attack VM pointed at an
uninstrumented estate teaches nothing. Both are met. What gates the build
now is its RAM kit and NVMe drive, in
now is its RAM kit and SATA SSD, in
[*Everything still to buy*](#everything-still-to-buy), and its place in the
#919 rack move.
→ [runbook](runbooks/build-the-playground.md)
Expand Down Expand Up @@ -347,8 +347,10 @@ ADR set; its last two rows are what it still needs. On 2026-10-13 the move
into the 32U rack ([#919](https://github.com/Gerrrt/HomeLab/issues/919)) put
its parts here, which had been listed only in its runbook's §0.1, and
`Saruman`'s memory joined them. Already on hand, so not listed: the frame,
the VESA mount, `trinity`'s ProDesk tray and the velcro, and since
2026-10-15 the keyboard drawer and `ifrit`'s Tiny mount, both mounted.
the VESA mount, `trinity`'s ProDesk tray and the velcro; since 2026-10-15
the keyboard drawer and `ifrit`'s Tiny mount, both mounted; and since
2026-10-16 the short patch leads in the VLAN colours and the spare M6 cage
nuts, which the operator already had.

| Item | For | Decided by | When it is needed |
| --- | --- | --- | --- |
Expand All @@ -357,15 +359,14 @@ the VESA mount, `trinity`'s ProDesk tray and the velcro, and since
| 1U horizontal cable manager | U20, between switch and panel | [`hardware.md`](hardware.md#planned-a-32u-rack) | Before the #919 window |
| 1U vented blank | U22, the air gap over the CRS326 | [`hardware.md`](hardware.md#planned-a-32u-rack) | Before the #919 window |
| 2 × pair of adjustable 1U four-post support rails, spanning 29" post to post. Rack Solutions 1UKIT-109 fits: 10–31.5", 45 lb, with a cabling crossbar | The keyboard drawer at U24 and `ifrit`'s Tiny tray at U18: both front-mounted, both wobble with every ear screw tight | The operator, 2026-10-15; the kit chosen 2026-10-16 | Before the #919 window |
| 2U four-post rail pair rated above the UPS's weight. Rack Solutions 2UKIT-109 fits: 10–31.75", 75 lb, 18-gauge, with a cabling crossbar | `mjolnir` at U1–U2: the rails it sits on now sag under it | The operator, 2026-10-16 | Before the #919 window |
| One factory-made Cat6 patch cable, long enough for the run | The XB7's striped 2.5G port to `morpheus`'s `em0`, replacing the hand-crimped patch | [#923](https://github.com/Gerrrt/HomeLab/issues/923) | Now: it should not wait for the move |
| Short factory-made Cat6 patch cables, a pack, in the ADR-0009 VLAN colours | Switch to patch panel in the new rack | [`move-into-the-32u-rack.md`](runbooks/move-into-the-32u-rack.md) §6.1 | Before the #919 window |
| Spare M6 cage nuts and screws, a pack | The frame | [`move-into-the-32u-rack.md`](runbooks/move-into-the-32u-rack.md) §0.1 | Before the #919 window |
| 8 × 16 GB DDR4 RDIMM, dual-rank, matched to a fitted stick's label (2Rx4 or 2Rx8, PC4-2400) | `Saruman`, 128 GB to 256 GB, one more per channel on both CPUs | The operator, 2026-10-13: headroom | In the #919 window, while the DL360 is out of the rack |
| 2 × 16 GB DDR4 SO-DIMM kit | `ifrit`, replacing the 8 GB it shipped with | [ADR-0017](adr/0017-buy-ifrit-for-iops-and-keep-the-range-disposable.md): 32 GB socketed | Before [`build-the-playground.md`](runbooks/build-the-playground.md) §1 |
| 1 TB NVMe SSD | `ifrit`'s range and its snapshot reverts | [ADR-0017](adr/0017-buy-ifrit-for-iops-and-keep-the-range-disposable.md): NVMe | Before [`build-the-playground.md`](runbooks/build-the-playground.md) §1 |
| 1 TB 2.5" SATA SSD | `ifrit`'s range and its snapshot reverts, in the Tiny's 2.5" bay; the 256 GB M.2 stays as the boot disk | [ADR-0096](adr/0096-fit-ifrits-range-disk-as-a-sata-ssd-in-the-tinys-bay.md), 2026-10-16 | Before [`build-the-playground.md`](runbooks/build-the-playground.md) §1 |

**The last one, bought 2026-10-05:** `ifrit`, the range host — a quiet
SFF box, NVMe, two socketed DIMM slots with 32 GB fitted, one NIC
SFF box with flash storage, two socketed DIMM slots with 32 GB fitted, one NIC
([ADR-0017](adr/0017-buy-ifrit-for-iops-and-keep-the-range-disposable.md),
[#421](https://github.com/Gerrrt/HomeLab/issues/421)). The box is a
refurbished ThinkCentre M80q, and it arrived on 2026-10-14. It shipped with
Expand Down
40 changes: 32 additions & 8 deletions docs/runbooks/build-the-playground.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,8 +43,10 @@ decision and on this runbook.

So what remains ahead of §1 is the machine. `ifrit` was bought on 2026-10-05,
a refurbished ThinkCentre M80q, and arrived on 2026-10-14. It is not ready
yet: ADR-0017 asks for 32 GB of socketed RAM and an NVMe disk, and the parts
chosen for that are still to fit. [`hardware.md`](../hardware.md) lists what
yet: ADR-0017 asks for 32 GB of socketed RAM and a flash disk, which
[ADR-0096](../adr/0096-fit-ifrits-range-disk-as-a-sata-ssd-in-the-tinys-bay.md)
makes a SATA SSD in the Tiny's bay, and the parts chosen for that are still
Comment thread
Gerrrt marked this conversation as resolved.
to fit. [`hardware.md`](../hardware.md) lists what
is still to buy and fit, and #421 tracks the build. Nothing below this section
can be started until they are in.

Expand All @@ -70,10 +72,30 @@ untagged VLAN 30. **No trunk, no tagged port** — ADR-0014 applies ADR-0007's
constraint on `Saruman` to `ifrit` by name, and a VLAN-aware bridge here is
listed as a reason to reopen that ADR.

Install Proxmox VE. Take the defaults; the only thing worth deciding at install
time is the filesystem, and there is one disk, so it is ext4 or a single-disk
ZFS root. ZFS is worth it here for one reason: the range's whole operating model
is snapshot and revert.
Install Proxmox VE **onto the 256 GB M.2, not the SATA SSD**
([ADR-0096](../adr/0096-fit-ifrits-range-disk-as-a-sata-ssd-in-the-tinys-bay.md)).
At the installer's target-disk step pick the M.2 by name, which it shows as
the `nvme` device, and leave the SATA drive unselected. ext4 is enough for a
root disk that holds nothing the range churns; the snapshots happen on the
other drive. Take the defaults otherwise.

Then make the SATA SSD the guest storage, because the installer does not.
From the node's shell, with the SSD's `by-id` name substituted:

```bash
ls -l /dev/disk/by-id/ | grep -i ata-
zpool create -o ashift=12 range /dev/disk/by-id/ata-<the 1 TB SSD>
pvesm add zfspool range --pool range --content images,rootdir --sparse 1
pvesm status
```

**You should see** `range` active beside `local` and the M.2's own
`local-lvm`. A single-disk pool with no redundancy is the point: ADR-0017
said "not a mirror", and the range is maintained by nobody. From here on,
**every template, the attack VM and every target gets its disk on `range`**:
pick it in the *Storage* field when creating each VM, and §6's `clean`
snapshot is then a ZFS snapshot on that pool. Nothing of the range goes on
the M.2.

Record what you actually bought in [`hardware.md`](../hardware.md) — the CPU,
the memory kit and the disk capacity are chosen at the till and written down
Expand Down Expand Up @@ -370,6 +392,7 @@ has never fired is a rule nobody knows can.

One VM, and it is the only guest with a leg on both bridges.

- **Its disk on `range`**, the SATA SSD's pool from §1, like every guest here.
- **`net0` on `vmbr0`** — ImaginationLAN, DHCP from the pool. Nothing points at
this address, so it does not get a reservation.
- **`net1` on `vmbr1`** — `172.30.30.10/24`, **no gateway**.
Expand Down Expand Up @@ -404,8 +427,9 @@ into the estate already need on `Saruman`.

## 6. The targets

Each target attaches to **`vmbr1` and nothing else**, takes a static address
from `172.30.30.100` upward, and has **no default route**. Every image insisting
Each target attaches to **`vmbr1` and nothing else**, keeps its disk on
**`range`** (§1), takes a static address from `172.30.30.100` upward, and has
**no default route**. Every image insisting
on DHCP gets a static override before its first boot on the bridge — ADR-0017
declines to run a DHCP server here, because the obvious place to run one is the
dual-homed VM, and a DHCP server that binds the wrong interface there is rogue
Expand Down
Loading
Loading