aws / aws/containers-roadmap

[EKS] [eks-node-monitoring-agent] Allow EKS Node Monitoring Agent to use existing DCGM installation

Open
#2,763 1 comment 10 reactions 0 assignees View on GitHub
EKS Nodes
Dominant language
Shell
Stars
5.4k
Forks
334
PR merge metrics
No merged PRs in 30d

Description

### Community Note

* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue or have submitted a pull request, please leave a comment

**Tell us about your request**
Allow the EKS Node Monitoring Agent (NMA) to use an existing installation of DCGM that was installed separately from NMA. This installation could be through direct install, GPU operator, or CloudWatch Observability Agent.

**Which service(s) is this request for?**
EKS

**Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?**
Users have reported issues when enabling NMA with an existing DCGM installation, where NMA reports AcceleratedHardwareReady = false with existing DCGM installation. This causes node provisioners such as Karpenter to terminate the nodes.

**Are you currently working around this issue?**
No workarounds fully satisfy the issue, other than using NMA built-in DCGM.

**Additional context**
See related issue #2555

Contributor guide

Open the contributing guide

Research direction

Start with related issue #2555 and trace how the EKS Node Monitoring Agent detects or uses DCGM when NMA is enabled. Compare the separately installed DCGM cases named here—direct install, GPU operator, and CloudWatch Observability Agent—and define done as NMA reporting AcceleratedHardwareReady=true without requiring its built-in DCGM.

Written by the indexing model from the issue text.

Assessment

Tech stack
kubernetes
Domain
cloud, infrastructure, observability-sre
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.