GPU Monitoring

GPU Monitoring GraphQL

Is your feature request related to a problem?

Yes, this feature request addresses critical gaps in the Unraid GraphQL API regarding GPU monitoring and management capabilities:

1. Missing GPU monitoring endpoints: The current Unraid GraphQL API lacks GPU monitoring queries, making it impossible for developers to build applications that monitor GPU performance, temperature, and utilization through the official API.

2. No GPU configuration mutations: There are no GraphQL mutations available for configuring GPU settings such as power limits, clock speeds, or fan curves, forcing developers to use external tools or direct system calls.

3. Absence of real-time GPU subscriptions: The GraphQL API doesn't provide subscription endpoints for real-time GPU monitoring, preventing developers from building responsive applications that react to GPU state changes.

4. Limited GPU resource management: API consumers cannot query GPU allocation status for VMs and containers, making it difficult to build resource management applications.

5. No standardized GPU data models: Without official GraphQL schemas for GPU data, developers must create their own inconsistent data structures, leading to fragmented ecosystem solutions.

6. Missing multi-vendor GPU support: The API doesn't provide a unified interface for different GPU vendors (NVIDIA, AMD, Intel), requiring separate implementations for each vendor.

Describe the solution you'd like

I would like comprehensive GPU monitoring capabilities added to the Unraid GraphQL API that provides:

GraphQL Schema Extensions

• GPU Data Types: Complete GraphQL type definitions for GPU devices, performance metrics, memory information, temperature data, and configuration options
• Resource Enum Extension: Addition of GPU_MONITORING to the existing Resource enum for permission management
• Input Types: Configuration input types for GPU settings, power limits, clock speeds, and fan control

Query Operations

• gpuMonitoring: Query to retrieve all GPU devices with current monitoring data including performance, temperature, memory usage, and process information
• gpuMonitoringById(id: String!): Query to get detailed information for a specific GPU device
• GPU resource allocation: Queries to show which VMs and containers are currently using specific GPUs
• Historical data access: Queries for GPU performance trends and usage history

Mutation Operations

• configureGpu(config: GpuConfigInput!): Mutation to configure GPU settings including power limits, clock speeds, and fan curves
• testGpuMonitoring(id: String!): Mutation to test GPU monitoring connectivity and validate configuration
• resetGpu(id: String!): Mutation to reset GPU settings to default values
• GPU allocation mutations: Assign/unassign GPUs to VMs and containers

Subscription Operations

• gpuUpdates: Real-time subscription for GPU performance updates including utilization, temperature, and power consumption
• gpuAlerts: Subscription for critical GPU alerts such as thermal warnings, power issues, or hardware errors
• gpuAllocationChanges: Subscription for GPU resource allocation changes in the system

Multi-Vendor API Support

• Unified GraphQL interface that abstracts vendor-specific differences between NVIDIA, AMD, and Intel GPUs
• Vendor detection: Automatic detection and appropriate tool selection (nvidia-smi, rocm-smi, intel-gpu-tools)
• Consistent data models: Standardized GraphQL types that work across all GPU vendors

Example API Usage

# Query GPU devices
query GetGPUMonitoring {
  info {
    gpuMonitoring {
      id
      devices {
        # Existing Gpu type fields
        id
        type              # Vendor (NVIDIA, AMD, Intel)
        vendorname        # Vendor name
        productid         # Product ID
        blacklisted       # VM allocation status
        
        # Extended monitoring fields
        monitoring {
          coreUtilization
          memoryUtilization
          coreClock
          memoryClock
          powerDraw
          fanSpeed
          performanceState
          lastUpdated
        }
        memory {
          total             # BigInt (following InfoMemory pattern)
          used              # BigInt
          free              # BigInt
          utilization       # Float percentage
          temperature       # Float
        }
        temperature {
          core
          memory
          hotspot
          unit              # Reuse existing TemperatureUnit enum
          throttling
        }
        processes {
          pid
          name
          memoryUsage       # BigInt
          gpuUtilization    # Float
        }
      }
      overallStatus
      lastUpdated
    }
  }
}

# Alternative: Query individual GPU devices
query GetGPUDevices {
  gpuDevices {
    id
    type
    vendorname
    monitoring {
      coreUtilization
      memoryUtilization
      powerDraw
    }
    temperature {
      core
      throttling
    }
  }
}

# Query specific GPU device by ID
query GetGPUDevice($id: String!) {
  gpuDeviceById(id: $id) {
    id
    type
    vendorname
    monitoring {
      coreUtilization
      memoryUtilization
      coreClock
      memoryClock
      powerDraw
      fanSpeed
    }
    memory {
      total
      used
      free
      utilization
    }
    temperature {
      core
      memory
      hotspot
      throttling
    }
  }
}

# Configure GPU settings
mutation ConfigureGPU($config: GpuConfigInput!) {
  configureGpu(config: $config) {
    # Return configuration object like UPS pattern
    service              # enable/disable
    monitoringInterval   # polling interval
    temperatureThresholds {
      warning
      critical
    }
    powerManagement {
      enabled
      powerLimit
    }
    # Success/error handling
    success
    message
    errors
  }
}

# Test GPU monitoring
mutation TestGPUMonitoring($id: String!) {
  testGpuMonitoring(id: $id) {
    success
    message
    testResults {
      monitoringAvailable
      vendorToolsDetected
      temperatureSensors
      performanceCounters
    }
  }
}

# Reset GPU settings
mutation ResetGPUSettings($id: String!) {
  resetGpuSettings(id: $id) {
    id
    type
    vendorname
    monitoring {
      coreUtilization
      powerDraw
    }
  }
}

# Subscribe to GPU updates
subscription GPUMonitoringUpdates {
  gpuMonitoringUpdates {
    id
    devices {
      id
      type
      monitoring {
        coreUtilization
        memoryUtilization
        powerDraw
        lastUpdated
      }
      temperature {
        core
        throttling
      }
    }
    overallStatus
    lastUpdated
  }
}

# GPU alerts subscription
subscription GPUAlerts {
  gpuAlerts {
    id
    timestamp
    deviceId
    alertType           # TEMPERATURE, POWER, UTILIZATION, ERROR
    severity           # WARNING, CRITICAL, EMERGENCY
    message
    currentValue
    threshold
    resolved
  }
}

# Individual GPU device updates (optional)
subscription GPUDeviceUpdates($deviceId: String) {
  gpuDeviceUpdates(deviceId: $deviceId) {
    id
    type
    monitoring {
      coreUtilization
      memoryUtilization
      powerDraw
    }
    temperature {
      core
      memory
      throttling
    }
  }
}

Additional context

GraphQL API Integration Details

The proposed GPU monitoring feature would integrate seamlessly with the existing Unraid GraphQL API:

• Type system integration: GPU monitoring types would follow existing Unraid GraphQL conventions and naming patterns
• Permission system: Uses existing Resource-based permissions with new GPU_MONITORING resource type
• Authentication: Leverages existing GraphQL authentication and authorization mechanisms
• Error handling: Follows established GraphQL error response patterns used throughout the Unraid API

Developer Use Cases

1. Dashboard Applications: Developers can build comprehensive system monitoring dashboards that include GPU metrics alongside existing system data
2. Mobile Applications: Real-time GPU monitoring apps using GraphQL subscriptions for live updates
3. Automation Tools: Scripts and applications that monitor GPU health and automatically adjust settings based on workload
4. Resource Management: Applications that track and allocate GPU resources across VMs and containers
5. Performance Analytics: Tools that analyze GPU performance trends and optimize system configurations

API Consumer Benefits

• Type Safety: Complete TypeScript/GraphQL type definitions for all GPU monitoring data
• Real-time Updates: WebSocket-based subscriptions for responsive applications
• Unified Interface: Single API endpoint for all GPU vendors and device types
• Query Flexibility: GraphQL's query language allows precise data fetching to minimize bandwidth
• Developer Experience: Consistent with existing Unraid GraphQL patterns and documentation

Implementation Considerations

• Backward Compatibility: New GraphQL types and fields won't affect existing API consumers
• Performance Impact: Efficient data fetching with configurable polling intervals and intelligent caching
• Schema Versioning: GPU monitoring additions follow Unraid's GraphQL schema versioning strategy
• Documentation: Complete GraphQL schema documentation with examples and best practices

Integration with Existing Unraid Features

• VM Management: GPU allocation status integrated with existing VM GraphQL queries
• Container Management: GPU resource tracking for Docker containers through existing container APIs
• User Management: GPU monitoring permissions integrated with existing user role system
• Notification System: GPU alerts integrated with existing Unraid notification GraphQL subscriptions

Environment (if relevant)

Unraid OS Version: 7.1.2 (requires existing GraphQL API infrastructure)

API Dependencies:

• Existing Unraid GraphQL API framework
• Current authentication and authorization system
• Established PubSub infrastructure for subscriptions
• Resource-based permission system

Hardware Requirements:

• One or more GPU devices (NVIDIA, AMD, or Intel)
• Sufficient system resources for GPU monitoring overhead
• Compatible GPU drivers installed on the system

Software Dependencies:

• NVIDIA drivers and nvidia-smi (for NVIDIA GPUs)
• AMD drivers and rocm-smi (for AMD GPUs)
• Intel GPU drivers and intel-gpu-tools (for Intel GPUs)
• Automatic vendor detection and tool selection

GraphQL Schema Requirements:

• Compatible with existing Unraid GraphQL schema versioning
• TypeScript code generation support for client applications
• Schema introspection support for development tools

Pre-submission Checklist

• I have searched existing issues to ensure this feature hasn't already been requested
• This is not an Unraid Connect related feature (if it is, please submit via the support form instead)
• I have provided clear examples or use cases for the feature
• I have focused specifically on GraphQL API extensions and capabilities
• I have considered alternative API approaches and explained why GraphQL is preferred
• I have provided detailed GraphQL schema examples and integration patterns
• I have specified API dependencies and compatibility requirements
• I have addressed how this integrates with existing Unraid GraphQL infrastructure

No files, tests, or entry points are named. Start by reviewing the existing GraphQL schema, Resource permissions, authentication, and PubSub subscription patterns; before coding, get maintainer agreement on the scope and acceptance criteria for monitoring, configuration, allocation, and multi-vendor support.