huggingface / huggingface/candle

Subject: Contribution for Metal Device Initialization Fix in Candle

Open
#3,369 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
21k
Forks
1.8k
Avg merge
16h 42m
Merged PRs (30d)
25

Description

By modifying the metal/device.rs code, I was able to successfully enable accelerated computation on an M1 Pro chip. I would like to submit these changes to you and have them incorporated into the repository, so that other developers can benefit from this improvement as well.

**-### code**
use crate::{Buffer, CommandQueue, ComputePipeline, Function, Library, MTLResourceOptions, MetalKernelError};

use std::{ffi::c_void, ptr};
use objc2_metal::{MTLCompileOptions, MTLCreateSystemDefaultDevice, MTLDevice};
use objc2_foundation::{NSArray, NSObject, NSString};
use objc2::{rc::Retained, runtime::ProtocolObject};
use objc2::{msg_send, msg_send_id, class, Message};
use objc2::rc::autoreleasepool;
use objc2::runtime::Object;

#[link(name = "Metal", kind = "framework")]
unsafe extern "C" {
unsafe fn MTLCopyAllDevices() -> *mut NSArray>;
}

/// Metal device type classification based on Apple Silicon architecture.
///
/// MLX uses the last character of the architecture name to determine device type:
/// - 'p': phone (iPhone, small device)
/// - 'g': base/pro (M1/M2/M3 base and Pro variants)
/// - 's': max (M1/M2/M3 Max)
/// - 'd': ultra (M1/M2 Ultra)
///
/// Reference: refs/mlx/mlx/backend/metal/device.cpp
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub enum MetalDeviceType {
/// Small device (iPhone, 'p' suffix)
Phone,
/// Base/Pro device (M1/M2/M3 base and Pro, 'g' suffix)
BasePro,
/// Max device (M1/M2/M3 Max, 's' suffix)
Max,
/// Ultra device (M1/M2 Ultra, 'd' suffix)
Ultra,
/// Unknown or medium device (default)
Medium,
}

#[derive(Clone, Debug)]
pub struct Device {
raw: Retained>,
}
unsafe impl Send for Device {}
unsafe impl Sync for Device {}

impl AsRef> for Device {
fn as_ref(&self) -> &ProtocolObject {
&self.raw
}
}

impl Device {
pub fn registry_id(&self) -> u64 {
self.as_ref().registryID()
}

pub fn all() -> Vec {
unsafe {
let array_ptr = MTLCopyAllDevices();
if array_ptr.is_null() {
return MTLCreateSystemDefaultDevice()
.map(|raw| vec![Device { raw }])
.unwrap_or_else(Vec::new);
}
let array = Retained::from_raw(array_ptr).expect("Failed to retain device array");
array
.iter()
.map(|raw_ref| { Device { raw: raw_ref.retain() } })
.collect()
}
}

pub fn system_default() -> Option {
MTLCreateSystemDefaultDevice().map(|raw| Device { raw })
}

pub fn new_buffer(
&self,
length: usize,
options: MTLResourceOptions,
) -> Result {
self.as_ref()
.newBufferWithLength_options(length, options)
.map(Buffer::new)
.ok_or(MetalKernelError::FailedToCreateResource(
"Buffer".to_string(),
))
}

pub fn new_buffer_with_data(
&self,
pointer: *const c_void,
length: usize,
options: MTLResourceOptions,
) -> Result {
let pointer = ptr::NonNull::new(pointer as *mut c_void).unwrap();
unsafe {
self.as_ref()
.newBufferWithBytes_length_options(pointer, length, options)
.map(Buffer::new)
.ok_or(MetalKernelError::FailedToCreateResource(
"Buffer".to_string(),
))
}
}

pub fn new_library_with_source(
&self,
source: &str,
options: Option<&MTLCompileOptions>,
) -> Result {
let raw = self
.as_ref()
.newLibraryWithSource_options_error(&NSString::from_str(source), options)
.unwrap();

Ok(Library::new(raw))
}

pub fn new_compute_pipeline_state_with_function(
&self,
function: &Function,
) -> Result {
let raw = self
.as_ref()
.newComputePipelineStateWithFunction_error(function.as_ref())
.unwrap();
Ok(ComputePipeline::new(raw))
}

pub fn new_command_queue(&self) -> Result {
let raw = self.as_ref().newCommandQueue().unwrap();
Ok(raw)
}

pub fn recommended_max_working_set_size(&self) -> usize {
self.as_ref().recommendedMaxWorkingSetSize() as usize
}

pub fn current_allocated_size(&self) -> usize {
self.as_ref().currentAllocatedSize()
}

/// Get the device architecture name (e.g., "applegpu_g13g", "applegpu_g14d").
///
/// This returns the full architecture string from the Metal device.
/// The last character indicates the device type:
/// - 'p': phone
/// - 'g': base/pro
/// - 's': max
/// - 'd': ultra
pub fn architecture_name(&self) -> String {
let arch = self.as_ref().architecture();
arch.name().to_string()
}

/// Get the device type based on architecture name.
///
/// This implements the same logic as MLX's device type detection.
/// Reference: refs/mlx/mlx/backend/metal/device.cpp
pub fn device_type(&self) -> MetalDeviceType {
let arch = self.architecture_name();
match arch.chars().last() {
Some('p') => MetalDeviceType::Phone,
Some('g') => MetalDeviceType::BasePro,
Some('s') => MetalDeviceType::Max,
Some('d') => MetalDeviceType::Ultra,
_ => MetalDeviceType::Medium,
}
}
}

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing metal/device.rs and comparing the current Metal device initialization with the implementation shown in the issue. Validate the behavior on an M1 Pro system, and consider the work done when accelerated computation initializes successfully without regressing other supported devices.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.