adam
0f0bdf4f6c
Prereqs verified on the cluster: VSO 1.3.0 (transformation supported, openbao-kubernetes VaultAuth healthy), MQTT egress to 10.5.30.3:1883 reachable from a pod, target nodes untainted, registry requires auth. Add registry-cred-vaultstaticsecret.yaml (secret/registry/nexus -> dockerconfigjson, mirroring existing cluster pattern) and wire imagePullSecrets into the DaemonSet. README updated with verified prereqs + apply order.
67 lines
2.8 KiB
Markdown
67 lines
2.8 KiB
Markdown
# jbod-monitor on RKE2 (DaemonSet)
|
|
|
|
Per-node self-contained deployment for Kubernetes worker nodes that can't run
|
|
the docker-compose stack. Each opted-in node runs one pod (`host-poller` +
|
|
`redis` + `web`) and publishes its own Home Assistant device
|
|
(`JBOD Monitor (<node>)`) over MQTT.
|
|
|
|
This mirrors the docker-compose stack with **no code changes** — each node is
|
|
independent, exactly like the standalone hosts.
|
|
|
|
## What it collects
|
|
|
|
- **host-poller** (privileged, hostPath `/dev`+`/sys`): IPMI/iDRAC env, CPU
|
|
coretemp, on-metal drive SMART → node-local redis.
|
|
- **web**: reads redis, publishes per-node MQTT discovery to the broker.
|
|
- No SAS poller (no SES enclosures on these nodes). Consequence: host-drive
|
|
entities have no ZFS pool label (no zfs_map producer).
|
|
|
|
## Prereqs — verified 2026-06-16
|
|
|
|
1. **Registry pull**: registry requires auth. `registry-cred-vaultstaticsecret.yaml`
|
|
syncs `secret/registry/nexus` → `docker-registry-cred` (dockerconfigjson),
|
|
referenced as `imagePullSecrets` in the DaemonSet. (Mirrors the cluster's
|
|
existing `*-registry-cred` pattern.)
|
|
2. **VSO**: v1.3.0; `openbao-kubernetes` VaultAuth healthy; `transformation.templates`
|
|
supported. No existing VSS in the cluster uses transformation, so watch the
|
|
`jbod-mqtt` VSS status on first apply for template errors.
|
|
3. **MQTT egress**: confirmed — a cluster pod reached `10.5.30.3:1883`.
|
|
4. **Nodes**: pascal/currie/fermi are untainted → no tolerations needed.
|
|
|
|
## Apply order
|
|
|
|
```bash
|
|
# 1. Label ONLY the intended nodes (do not blanket-label all workers):
|
|
kubectl label node usco-dc-pascal usco-dc-currie usco-dc-fermi jbod-monitor=enabled
|
|
# (canary: label just usco-dc-fermi first)
|
|
|
|
# 2. Namespace + DaemonSet:
|
|
kubectl apply -f k8s/daemonset.yaml
|
|
|
|
# 3. Secrets (VSO → docker-registry-cred + jbod-mqtt). Apply, then confirm sync:
|
|
kubectl apply -f k8s/registry-cred-vaultstaticsecret.yaml
|
|
kubectl apply -f k8s/mqtt-vaultstaticsecret.yaml
|
|
kubectl -n jbod-monitor get vaultstaticsecret # SYNCED=True for both
|
|
kubectl -n jbod-monitor get secret jbod-mqtt -o jsonpath='{.data.MQTT_USERNAME}' | base64 -d
|
|
|
|
# 4. Roll the DaemonSet so pods pick up the secrets if they raced ahead:
|
|
kubectl -n jbod-monitor rollout restart daemonset/jbod-monitor
|
|
```
|
|
|
|
## Verify
|
|
|
|
```bash
|
|
kubectl -n jbod-monitor get pods -o wide # one per labeled node
|
|
kubectl -n jbod-monitor logs <pod> -c host-poller | grep sweep
|
|
kubectl -n jbod-monitor logs <pod> -c web | grep -i mqtt # "MQTT connected"
|
|
```
|
|
|
|
Then check Home Assistant for `JBOD Monitor (usco-dc-pascal|currie|fermi)`.
|
|
|
|
## Notes
|
|
|
|
- pascal (R620) already has an iDRAC "System Board Inlet Temp" in HA via another
|
|
integration — its env temp will partially duplicate; drives + CPU are net-new.
|
|
- Pin image tags by SHA (already done: host-poller `514502e`, web `1839e9d`).
|
|
- Removal: `kubectl delete -f k8s/daemonset.yaml` and unlabel the nodes.
|