jbod-monitor/README.md

JBOD Monitor

Drive-health monitoring for JBOD enclosures on Linux, with a REST API, web UI, and optional Home Assistant MQTT publishing.

Auto-discovers SES enclosures via sysfs, maps drives to physical slots, reads SMART, and aggregates per-enclosure temperatures (named SES sensors + hotspot).

Architecture

Two processes, with Redis as the only interface between them:

poller (privileged, pid:host)  ──smartctl/sg_ses/zpool/ledctl──>  hardware
        │  serialized + paced
        ▼
      Redis  ◀── reads ──  app (REST API + web UI + MQTT)   [unprivileged]
        ▲
        └── locate-LED command queue (app enqueues, poller runs)

• poller is the only process that touches the SAS bus. All hardware commands run behind a single gate (POLL_CONCURRENCY, default 1 = fully serialized) with a gap between drives, so it never storms fragile SAS expanders. It sweeps on an interval and writes results to Redis.
• app (and the MQTT publisher) are pure Redis readers — unprivileged, no /dev. They can restart freely without issuing a single hardware command.
• Locate LEDs: the API enqueues a request in Redis; the poller executes it serialized with everything else.

This split exists for a reason: concurrent/bursty SMART+SES traffic — especially repeated on every container restart — can drop SAS expanders and suspend pools. One paced owner makes that structurally impossible.

Prerequisites

• Linux with SAS/SATA JBODs connected via HBA
• smartmontools (smartctl), sg3-utils (sg_ses), ledmon (ledctl)
• Redis
• Python 3.11+

apt install smartmontools sg3-utils ledmon   # Debian/Ubuntu

Run (docker compose)

docker compose up -d --build

Brings up poller (privileged), app (port 8000), and redis. To run the two processes by hand instead:

REDIS_HOST=localhost sudo -E python poller.py            # producer (needs root)
REDIS_HOST=localhost uvicorn main:app --port 8000        # consumer

API Endpoints

Endpoint	Description
GET /api/health	Service health + redis/mqtt/poller_fresh status
GET /api/enclosures	List discovered SES enclosures
GET /api/enclosures/{id}/drives	Drive slots for an enclosure
GET /api/drives/{device}	SMART detail for a block device
GET /api/overview	Aggregate enclosure + drive health
GET /api/temps	Per-enclosure temperatures: named SES sensors + hotspot
POST /api/drives/{device}/led	Queue a locate-LED change (state: locate/off)
GET /docs	Swagger UI

Read endpoints serve the poller's last sweep; X-Poll-Age (seconds) headers and the poller_fresh health flag surface staleness.

Poller tuning (env)

Variable	Default	Description
POLL_SWEEP_INTERVAL	300	Seconds between full sweeps
POLL_DRIVE_GAP	0.5	Seconds between each drive's SMART query
POLL_CONCURRENCY	1	Max concurrent hardware ops (1 = serialized)
POLL_STARTUP_JITTER	10	Random startup delay to avoid restart storms
POLL_DATA_TTL	900	Redis TTL for poller data (must exceed sweep interval)
ZFS_USE_NSENTER	—	true to run zpool in the host namespace (containers)

Home Assistant (MQTT)

The app publishes per-enclosure temperatures via MQTT discovery — no HA YAML. Each enclosure becomes a device with a Hotspot sensor (max across SES sensors + housed drive temps; drive_max_c/drive_temp_count as attributes) and one sensor per named SES temperature element. Opt-in via MQTT_HOST; availability is tracked via an MQTT LWT.

Variable	Default	Description
MQTT_HOST	(unset)	Broker host. Unset → MQTT disabled.
MQTT_PORT	1883	Broker port
MQTT_USERNAME / MQTT_PASSWORD	(unset)	Broker credentials
MQTT_DISCOVERY_PREFIX	homeassistant	HA discovery topic prefix
MQTT_BASE_TOPIC	jbod-monitor	State/availability topic root
MQTT_PUBLISH_INTERVAL	60	Seconds between publishes
MQTT_NODE_ID	(hostname)	Stable id (disambiguates multiple hosts)

Credentials: simplest is a root-owned /etc/jbod-monitor/mqtt.env with MQTT_USERNAME=/MQTT_PASSWORD=, loaded via compose env_file. Optionally the app can read them from OpenBao at runtime (MQTT_SECRET_PATH + OPENBAO_*, keys via MQTT_SECRET_USER_KEY/MQTT_SECRET_PASS_KEY) — see services/secrets.py; it falls back to the env vars if the read fails.

Topics:

homeassistant/sensor/<node>_enc<id>/hotspot/config   # retained discovery
homeassistant/sensor/<node>_enc<id>/temp<n>/config    # retained discovery
jbod-monitor/<node>/status                            # online | offline (LWT)
jbod-monitor/<node>/enclosure/<id>/state              # {"hotspot_c":42.0, ...}