Incident

pfSense DHCP Outage a day after installing NetGate Plus 26.03-RELEASE

Summary

On 2026-04-06 at approximately 09:38 EDT, the Kea DHCP service on pfSense (26.03-RELEASE) stopped serving leases on all interfaces. The root cause was a corruption of the <dhcpd> block in /cf/conf/config.xml, which caused pfSense to generate an empty Kea interfaces list. Service was restored by activating a previous ZFS boot environment.


Timeline

Time (EDT) Event
08:54 config.xml drops ~15KB — <dhcpd> block wiped (config-1775480086.xml > config-1775480087.xml)
08:54–09:37 Multiple config saves at reduced size; pfSense regenerates Kea config with empty interfaces each time
09:38 Kea begins logging DHCPSRV_NO_SOCKETS_OPEN; DHCP stops serving leases
09:38–09:41 Kea restarts repeatedly, fails each time
~09:45 Operator detects outage; accesses box via WireGuard VPN
~10:00 Investigation begins
~10:30 Root cause identified: empty <dhcpd> in config.xml
~10:45 Config restore from backup attempted; Kea config generator continues producing empty interfaces
~11:00 ZFS boot environment rollback to default_20260405014509 initiated
~11:10 Service restored

Root Cause

A pfSense package operation (pfBlockerNG or Suricata reload/apply) triggered a config write at 08:54 that silently cleared the entire <dhcpd> section from /cf/conf/config.xml. This is a pfSense 26.x bug: package-initiated config saves can clobber unrelated service configuration blocks.

 …

TrueNAS reboot loop, VM load, and the NVMe that would not stay seated

Overview

This is the build-and-break log for bringing a previously unstable TrueNAS/FreeBSD box back to a usable state after months of ugly crash behavior under VM load.

The short version:

  • Salvaged hardware repurposed as a NAS
  • A Fedora VM running Plex-related work could trigger a reboot loop
  • Early on, the machine was already rebooting itself in a loop — even before the NVMe path failure showed up clearly in logs
  • Hard evidence eventually pointed at one NVMe path (nvme2 / nda2) timing out and detaching under load
  • Reseating the suspect SN770 and replacing its heatsink brought the mirror back online
  • Current best hypothesis: physical seating, thermal contact, or mechanical instability — not a software bug
 …

Removing a Dead iCloud Account on a Pre-T2 Mac

(Mac Pro 2013 / Non-Secure Enclave Macs)


⚠️ Scope

This guide applies only to pre-T2 Macs (e.g., Mac Pro Late 2013).

These machines:

  • Do NOT have a T2 chip
  • Do NOT enforce hardware Activation Lock
  • Store iCloud / Find My bindings in macOS user & system databases

This will remove the local iCloud association without deleting the Apple ID account itself.


🧠 Understanding the Architecture

On pre-T2 Macs:

  • iCloud login state is stored locally in user databases
  • Find My relies on account tokens
  • There is no firmware-level enforcement

Removing the local databases clears the association from macOS.

 …