inventory-agent: deliver and schedule it on the node
- Dominant language
- Go
- Stars
- 28
- Forks
- 11
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 55
Description
`inventory-agent` is built and released but nothing installs it, configures it, or
runs it. It is the only inventory component with no deployment path at all, which
means the aggregator, viewer and inspector have nothing to aggregate.
## Why it is a host process, not a workload
`designs/inventory.md` says "The agent runs as root on each node", and lists
deployment manifests for the aggregator, viewer and inspector only. The code
agrees, and rules out a DaemonSet as currently written:
- **One-shot.** `Execute()` (`internal/inventory/agent/collect.go:104-136`) collects,
writes SQLite, publishes over gRPC and returns. No loop, no interval flag. A
DaemonSet pod can only use `restartPolicy: Always`, so a cleanly-exiting binary
would be restarted forever.
- **Root required.** `collect.go:105-107` hard-fails when `os.Geteuid() != 0`.
- **Deep host access.** `/dev/ipmi0`, the 0400 `product_serial` and `product_uuid`
under `/sys/class/dmi/id`, `/proc/cpuinfo`, `/proc/meminfo`.
- **`--db ./inventory.db`** is relative to the working directory; the design calls
it a node cache.
#633 accordingly ships it as a GoReleaser binary and does not build an image for
it.
## Work
**1. systemd service plus timer.** The one-shot, root-only shape fits a timer
exactly. Precedent for unit assets the node agent installs:
`pkg/agent/phases/rootfs/assets/localdns.service`, `config-regeneration.service`,
`pkg/agent/phases/nodestart/assets/kubelet.service`.
**2. Binary delivery.** Either through the node agent's goalstate machinery
(`pkg/agent/goalstates/agentupgrade.go` shows the blue-green pattern for the
agent's own binary) or baked into `images/host-ubuntu2404`. Whichever is chosen has
to handle upgrade alongside the rest of the node's software.
**3. Collector tool packages** via `pkg/agent/phases/host/apt.go`. Each backs a
specific collector, and a missing binary means that collector silently reports
nothing:
| Package | Collector |
|---|---|
| `dmidecode` | `cpu.go`, `memory.go` - DMI processor and DIMM tables |
| `pciutils` | `gpu.go` - `lspci` enumeration |
| `ethtool` | `network.go` - NIC driver and firmware |
| `kmod` | `disk.go`, `gpu.go` - `modinfo` driver versions |
| `udev` | `disk.go` - `udevadm` disk properties |
| `lldpd` | `lldp.go` - `lldpctl` switch neighbours |
The NVIDIA tools (`nvidia-smi`, `nvidia-imex-ctl`) come from the host driver and
are version-locked to it; they cannot be installed here.
**4. The collector endpoint.** `--collector` defaults to
`inventory-collector:50051` (`cmd/inventory/inventory-agent/main.go:28`), a cluster
Service DNS name that cannot resolve from the host. It needs a NodePort,
LoadBalancer, hostNetwork Service or an explicitly configured address, and that
address has to reach each node as config. This couples to #636, where the Service
becomes `inventory-collector-`.
**5. `--db` path.** `./inventory.db` should become something durable such as
`/var/lib/unbounded/inventory.db` rather than depending on the unit's working
directory.
**6. Cadence and failure handling.** At provisioning only, or periodically, and
with what jitter. What happens when the aggregator is unreachable: the local SQLite
cache already exists, so the question is whether the agent retries, or the next
timer firing simply re-publishes.
## Open question: how to prove it works
This is where a true end-to-end test becomes possible, and it is also where it is
hardest. The collectors read DMI, IPMI, PCI, LLDP and NVLink, which return little
or nothing on a cloud VM, so a meaningful test needs real hardware - pointing at a
bare-metal site such as `boulderlab`, which is also the site that goes unreachable
and would need the tolerance treatment the DaemonSet deploy gate already has.
Cheaper options worth considering first: assert the unit is installed and enabled
after provisioning; assert one timer firing produces a non-empty local SQLite
database; assert a record for the node's serial appears in PostgreSQL within one
interval.
## Acceptance criteria
- After a node is provisioned, `inventory-agent` is installed, its unit is enabled,
and it has run at least once.
- A record identified by the node's DMI serial reaches the aggregator.
- The agent survives an unreachable aggregator without wedging the node or filling
the disk.
- The collector address is configuration, not a hardcoded cluster Service name.
Contributor guide
Research direction
Start with designs/inventory.md, internal/inventory/agent/collect.go, cmd/inventory/inventory-agent/main.go, the unit assets under pkg/agent/phases, and pkg/agent/phases/host/apt.go. Compare the delivery and provisioning precedents, then define how configuration and cadence are represented; done means the agent is installed and runs, publishes the node record, tolerates an unreachable aggregator, and has verification for the stated acceptance criteria.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kubernetes, postgresql, sqlite
- Domain
- databases, devops, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100