ROS2 wrapper for DA3?

Have you all started on an ROS2 Wrapper for this project? If not, I've made progress on one here: https://github.com/GerdsenAI/GerdsenAI-Depth-Anything-3-ROS2-Wrapper.

`# WORK IN PROGRESS LOOKING FOR CONTRIBUTERS :)

# Depth Anything 3 ROS2 Wrapper

## Acknowledgments and Credits

This package would not be possible without the excellent work of the following projects and teams:

### Depth Anything 3
- **Team**: ByteDance Seed Team
- **Repository**: [ByteDance-Seed/Depth-Anything-3](https://github.com/ByteDance-Seed/Depth-Anything-3)
- **Paper**: [Depth Anything 3: A New Foundation for Metric and Relative Depth Estimation](https://arxiv.org/abs/2511.10647)
- **Project Page**: https://depth-anything-3.github.io/

This wrapper integrates the state-of-the-art Depth Anything 3 model for monocular depth estimation. All credit for the model architecture and training goes to the original authors.

### Inspiration from Prior ROS2 Wrappers
This package was inspired by the following excellent ROS2 wrapper implementations:

- **Depth Anything V2 ROS2**: [grupo-avispa/depth_anything_v2_ros2](https://github.com/grupo-avispa/depth_anything_v2_ros2)
- **Depth Anything ROS2**: [polatztrk/depth_anything_ros](https://github.com/polatztrk/depth_anything_ros)
- **TensorRT Optimized Wrapper**: [scepter914/DepthAnything-ROS](https://github.com/scepter914/DepthAnything-ROS)

Special thanks to these developers for demonstrating effective patterns for ROS2 integration.

---

## Overview

This aims to be a camera-agnostic ROS2 wrapper for Depth Anything 3 (DA3), providing real-time monocular depth estimation from standard RGB images. This package is designed to work seamlessly with any camera publishing standard `sensor_msgs/Image` messages.

### Key Features

- **Camera-Agnostic Design**: Works with ANY camera publishing standard ROS2 image topics
- **Multiple Model Support**: All DA3 variants (Small, Base, Large, Giant, Nested)
- **CUDA Acceleration**: Optimized for NVIDIA GPUs with automatic CPU fallback
- **Multi-Camera Support**: Run multiple instances for multi-camera setups
- **Real-Time Performance**: Optimized for low latency on Jetson Orin AGX
- **Production Ready**: Comprehensive error handling, logging, and testing
- **RViz2 Visualization**: Pre-configured visualization setup

### Supported Platforms

- **Primary**: NVIDIA Jetson Orin AGX 64GB (JetPack 6.x)
- **Compatible**: Any system with Ubuntu 22.04, ROS2 Humble, and CUDA 12.x (or CPU)
- **ROS2 Distribution**: Humble Hawksbill
- **Python**: 3.10+

---

## Table of Contents

- [Installation](#installation)
- [Quick Start](#quick-start)
- [Configuration](#configuration)
- [Usage Examples](#usage-examples)
- [Performance](#performance)
- [Troubleshooting](#troubleshooting)
- [Development](#development)
- [Citation](#citation)
- [License](#license)

---

## Installation

### Prerequisites

1. **ROS2 Humble** on Ubuntu 22.04:
```bash
# If not already installed
sudo apt update
sudo apt install ros-humble-desktop
```

2. **CUDA 12.x** (optional, for GPU acceleration):
```bash
# For Jetson Orin AGX, this comes with JetPack 6.x
# For desktop systems, install CUDA Toolkit from NVIDIA
nvidia-smi # Verify CUDA installation
```

### Step 1: Install ROS2 Dependencies

```bash
sudo apt install -y \
ros-humble-cv-bridge \
ros-humble-sensor-msgs \
ros-humble-std-msgs \
ros-humble-image-transport \
ros-humble-rclpy
```

### Step 2: Install Python Dependencies

```bash
# Create and activate a virtual environment (recommended)
python3 -m venv ~/da3_venv
source ~/da3_venv/bin/activate

# Install PyTorch with CUDA support
pip3 install torch torchvision --index-url https://download.pytorch.org/whl/cu121

# Install other dependencies
pip3 install transformers>=4.35.0 \
huggingface-hub>=0.19.0 \
opencv-python>=4.8.0 \
pillow>=10.0.0 \
numpy>=1.24.0 \
timm>=0.9.0

# Install Depth Anything 3 from source
pip3 install git+https://github.com/ByteDance-Seed/Depth-Anything-3.git
```

**Note**: For CPU-only systems, install PyTorch without CUDA:
```bash
pip3 install torch torchvision --index-url https://download.pytorch.org/whl/cpu
```

### Step 3: Clone and Build Package

```bash
# Navigate to your ROS2 workspace
cd ~/ros2_ws/src # Or create: mkdir -p ~/ros2_ws/src && cd ~/ros2_ws/src

# Clone this repository
git clone https://github.com/yourusername/depth_anything_3_ros2.git

# Build the package
cd ~/ros2_ws
colcon build --packages-select depth_anything_3_ros2

# Source the workspace
source install/setup.bash
```

### Step 4: Verify Installation

```bash
# Test that the package is found
ros2 pkg list | grep depth_anything_3_ros2

# Run tests (optional)
colcon test --packages-select depth_anything_3_ros2
colcon test-result --verbose
```

---

## Quick Start

### Single Camera (Generic USB Camera)

The fastest way to get started is with a standard USB camera:

```bash
# Terminal 1: Launch USB camera driver
ros2 run v4l2_camera v4l2_camera_node --ros-args \
-p image_size:="[640,480]" \
-r __ns:=/camera

# Terminal 2: Launch Depth Anything 3
ros2 launch depth_anything_3_ros2 depth_anything_3.launch.py \
image_topic:=/camera/image_raw \
model_name:=depth-anything/DA3-BASE \
device:=cuda

# Terminal 3: Visualize with RViz2
rviz2 -d $(ros2 pkg prefix depth_anything_3_ros2)/share/depth_anything_3_ros2/rviz/depth_view.rviz
```

### Using Pre-Built Example Launch Files

```bash
# USB camera example (requires v4l2_camera)
ros2 launch depth_anything_3_ros2 usb_camera_example.launch.py

# Static image test (requires image_publisher)
ros2 launch depth_anything_3_ros2 image_publisher_test.launch.py \
image_path:=/path/to/your/test_image.jpg
```

---

## Configuration

### Parameters

All parameters can be configured via launch files or command line:

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `model_name` | string | `depth-anything/DA3-BASE` | Hugging Face model ID or local path |
| `device` | string | `cuda` | Inference device (`cuda` or `cpu`) |
| `cache_dir` | string | `""` | Model cache directory (empty for default) |
| `inference_height` | int | `518` | Height for inference (model input) |
| `inference_width` | int | `518` | Width for inference (model input) |
| `input_encoding` | string | `bgr8` | Expected input encoding (`bgr8` or `rgb8`) |
| `normalize_depth` | bool | `true` | Normalize depth to [0, 1] range |
| `publish_colored` | bool | `true` | Publish colorized depth visualization |
| `publish_confidence` | bool | `true` | Publish confidence map |
| `colormap` | string | `turbo` | Colormap for visualization |
| `queue_size` | int | `1` | Subscriber queue size |
| `log_inference_time` | bool | `false` | Log performance metrics |

### Available Models

| Model | Parameters | Use Case |
|-------|------------|----------|
| `depth-anything/DA3-SMALL` | 0.08B | Fast inference, lower accuracy |
| `depth-anything/DA3-BASE` | 0.12B | Balanced performance (recommended) |
| `depth-anything/DA3-LARGE` | 0.35B | Higher accuracy |
| `depth-anything/DA3-GIANT` | 1.15B | Best accuracy, slower |
| `depth-anything/DA3NESTED-GIANT-LARGE` | Combined | Metric scale reconstruction |

### Topics

#### Subscribed Topics
- `~/image_raw` (sensor_msgs/Image): Input RGB image from camera
- `~/camera_info` (sensor_msgs/CameraInfo): Optional camera intrinsics

#### Published Topics
- `~/depth` (sensor_msgs/Image): Depth map (32FC1 encoding)
- `~/depth_colored` (sensor_msgs/Image): Colorized depth visualization (BGR8)
- `~/confidence` (sensor_msgs/Image): Confidence map (32FC1)
- `~/depth/camera_info` (sensor_msgs/CameraInfo): Camera info for depth image

---

## Usage Examples

### Example 1: Generic USB Camera (v4l2_camera)

Complete example with a standard USB webcam:

```bash
# Install v4l2_camera if not already installed
sudo apt install ros-humble-v4l2-camera

# Launch everything together
ros2 launch depth_anything_3_ros2 usb_camera_example.launch.py \
video_device:=/dev/video0 \
model_name:=depth-anything/DA3-BASE
```

### Example 2: ZED Stereo Camera

Connect to a ZED camera (requires separate ZED ROS2 wrapper installation):

```bash
# Launch ZED camera separately
ros2 launch zed_wrapper zed_camera.launch.py camera_model:=zedxm

# In another terminal, launch depth estimation with topic remapping
ros2 launch depth_anything_3_ros2 depth_anything_3.launch.py \
image_topic:=/zed/zed_node/rgb/image_rect_color \
camera_info_topic:=/zed/zed_node/rgb/camera_info
```

Or use the provided example:
```bash
ros2 launch depth_anything_3_ros2 zed_camera_example.launch.py \
camera_model:=zedxm
```

### Example 3: Intel RealSense Camera

Connect to a RealSense camera (requires realsense-ros):

```bash
# Launch RealSense camera
ros2 launch realsense2_camera rs_launch.py

# Launch depth estimation
ros2 launch depth_anything_3_ros2 realsense_example.launch.py
```

### Example 4: Multi-Camera Setup

Run depth estimation on 4 cameras simultaneously:

```bash
# Launch multi-camera setup
ros2 launch depth_anything_3_ros2 multi_camera.launch.py \
camera_namespaces:="cam1,cam2,cam3,cam4" \
image_topics:="/cam1/image_raw,/cam2/image_raw,/cam3/image_raw,/cam4/image_raw" \
model_name:=depth-anything/DA3-BASE
```

### Example 5: Testing with Static Images

Test with a static image using image_publisher:

```bash
sudo apt install ros-humble-image-publisher

ros2 launch depth_anything_3_ros2 image_publisher_test.launch.py \
image_path:=/path/to/test_image.jpg \
model_name:=depth-anything/DA3-BASE
```

### Example 6: Using Different Models

Switch between models for different performance/accuracy tradeoffs:

```bash
# Fast inference (DA3-Small)
ros2 launch depth_anything_3_ros2 depth_anything_3.launch.py \
model_name:=depth-anything/DA3-SMALL \
image_topic:=/camera/image_raw

# Best accuracy (DA3-Giant) - requires more GPU memory
ros2 launch depth_anything_3_ros2 depth_anything_3.launch.py \
model_name:=depth-anything/DA3-GIANT \
image_topic:=/camera/image_raw
```

### Example 7: CPU-Only Mode

Run on systems without CUDA:

```bash
ros2 launch depth_anything_3_ros2 depth_anything_3.launch.py \
image_topic:=/camera/image_raw \
model_name:=depth-anything/DA3-BASE \
device:=cpu
```

### Example 8: Custom Configuration

Use a custom parameter file:

```bash
# Create custom config file
cat > my_config.yaml <