Split into dedicated hardware poller + read-only consumers

Rearchitect so a single poller process is the sole owner of all SAS/SMART/
SES hardware I/O, serialized behind one gate and paced, writing results to
Redis. The API/web and MQTT publisher become pure Redis readers — they no
longer issue any subprocess and can restart freely without touching the bus.

This addresses backplane/expander stress from concurrent + restart-triggered
SMART/SES storms (the prior model re-ran a hardware sweep on every container
start, and polled sg_ses 0x02+0x07 every 60s; 0x07 errored on the IOM6/
Xyratex expanders).

- poller.py: paced sweep (inventory, SMART per-drive w/ gap, SES 0x02, ZFS,
  host/MegaRAID), startup jitter, single-instance Redis lock, LED-queue worker
- services/hwgate.py: global serialization semaphore (POLL_CONCURRENCY=1)
- services/store.py: Redis as the only producer<->consumer interface + LED queue
- services/health.py: shared drive-health classifier (fixes overview double-count)
- gate smartctl/sg_ses/zpool/ledctl; drop sg_ses 0x07 from the hot path
- routers + temps read-only from the store; main.py drops the in-process poller
- compose: 3 services (privileged poller + unprivileged app + redis) w/ pacing knobs

docker-compose.yml

@@ -1,43 +1,59 @@
 services:
-  jbod-monitor:
+  # Producer: the ONLY service that touches hardware (smartctl/sg_ses/zpool/
+  # ledctl). Serialized + paced; writes everything to Redis.
+  poller:
     build: .
     image: docker.adamksmith.xyz/jbod-monitor:latest
-    container_name: jbod-monitor
+    container_name: jbod-poller
     restart: unless-stopped
     privileged: true
     pid: host
     network_mode: host
+    command: ["python", "-u", "poller.py"]
     volumes:
       - /dev:/dev
       - /sys:/sys:ro
       - /run/udev:/run/udev:ro
-      # OpenBao RO token (claude-read) mounted read-only; never bake into image.
-      - /etc/jbod-monitor/openbao-token:/run/secrets/openbao-token:ro
     environment:
       - TZ=America/Denver
-      - UVICORN_LOG_LEVEL=info
       - ZFS_USE_NSENTER=true
       - REDIS_HOST=127.0.0.1
       - REDIS_PORT=6379
       - REDIS_DB=0
-      - SMART_CACHE_TTL=120
-      - SMART_POLL_INTERVAL=90
-      # Home Assistant MQTT publishing (opt-in: leave MQTT_HOST unset to disable).
-      # EMQX at 10.5.30.3 is reachable by IP and bridged to the Mosquitto HA
-      # add-on, so discovery reaches Home Assistant either way.
+      # Pacing — keep the SAS bus quiet. POLL_CONCURRENCY=1 fully serializes.
+      - POLL_SWEEP_INTERVAL=300
+      - POLL_DRIVE_GAP=0.5
+      - POLL_CONCURRENCY=1
+      - POLL_STARTUP_JITTER=10
+      - POLL_DATA_TTL=900
+    depends_on:
+      - redis
+
+  # Consumer: REST API + web UI + Home Assistant MQTT. Reads Redis only —
+  # unprivileged, no /dev, can restart freely without touching hardware.
+  app:
+    build: .
+    image: docker.adamksmith.xyz/jbod-monitor:latest
+    container_name: jbod-monitor
+    restart: unless-stopped
+    network_mode: host
+    environment:
+      - TZ=America/Denver
+      - UVICORN_LOG_LEVEL=info
+      - REDIS_HOST=127.0.0.1
+      - REDIS_PORT=6379
+      - REDIS_DB=0
+      # Home Assistant MQTT publishing (EMQX, bridged to Mosquitto HA add-on).
       - MQTT_HOST=10.5.30.3
       - MQTT_PORT=1883
       - MQTT_DISCOVERY_PREFIX=homeassistant
       - MQTT_BASE_TOPIC=jbod-monitor
       - MQTT_PUBLISH_INTERVAL=60
-      # Broker credentials sourced from OpenBao (secret/home_assistant,
-      # keys: mqtt_user/mqtt_pass). Falls back to MQTT_USERNAME/MQTT_PASSWORD
-      # env if the read fails. Token is the read-only claude-read token.
-      - OPENBAO_ADDR=https://vault.adamksmith.xyz
-      - OPENBAO_TOKEN_FILE=/run/secrets/openbao-token
-      - MQTT_SECRET_PATH=home_assistant
-      - MQTT_SECRET_USER_KEY=mqtt_user
-      - MQTT_SECRET_PASS_KEY=mqtt_pass
+    # Broker credentials (MQTT_USERNAME/MQTT_PASSWORD). Optional: container
+    # starts without it (MQTT just won't authenticate).
+    env_file:
+      - path: /etc/jbod-monitor/mqtt.env
+        required: false
     depends_on:
       - redis