ashvardanian / ashvardanian/NumKong

Tracking: N-D numerical, image, and volume kernels requested by AlbumentationsX

Open
#313 1 comment 6 reactions 0 assignees View on GitHub
enhancement
Dominant language
C
Stars
1.9k
Forks
130
Avg merge
18h 28m
Merged PRs (30d)
3

Description

## Goal

Track the NumKong primitives that would let AlbumentationsX and Albucore process images, batches, videos, and volumes without Python loops or chains of NumPy temporaries.

The common layouts are channel-last:

- Image: `(H, W, C)`.
- Image batch or video: `(N, H, W, C)`.
- Volume: `(D, H, W, C)`.

## Numerical Tensor layer

- #368 — buffer-first N-D cast with `out=` and explicit conversion semantics.
- #311 — axis-aware `moments` and `minmax`.
- #315 — mixed-dtype broadcasted affine transform for normalization.
- #327 — shape-preserving N-D `scale` and explicit integer rounding.
- #366 — broadcasted comparisons, `minimum`, `maximum`, `clip`, `where`, and fused compare-select.
- #369 — N-D `lerp` with scalar, channel, image, batch, or volume weight tensors.
- #365 — elementary math including `abs`, `sqrt`, `exp`, `log`, `floor`, `ceil`, `rint`, `atan2`, and `hypot`.
- #363 — histogram and `bincount`.
- #247 — N-D `take`/gather for channel permutations.
- #312 — direct `matmul` for one-shot and changing matrices.
- #299 — stateful SIMD random generators.

## Spatial and neighborhood layer

- #362 — batched 2D and true 3D resampling for resize, affine/perspective warp, and dense remap.
- #364 — N-D `convolve1d` and separable box/Gaussian filters.
- #367 — N-D minimum, maximum, median, and mode neighborhood filters; morphology derives from min/max filters.

## Current 3D dependency chain

AlbumentationsX is expanding from slice-wise volume handling to true 3D augmentation. For that work, #362 and #364 are the two foundational kernels rather than general future optimizations:

- #362 unlocks `Resize3D`, `Affine3D`, `ElasticTransform3D`, and `Anisotropy3D`.
- #364 unlocks true `GaussianBlur` over D/H/W and smoothing for 3D elastic displacement fields.
- #367 later unlocks true 3D morphology and median/mode neighborhood transforms.

Equivalent OpenCV requests are tracked in [opencv/opencv#29605](https://github.com/opencv/opencv/issues/29605) and [opencv/opencv#29604](https://github.com/opencv/opencv/issues/29604). Having both requests lets Albucore benchmark semantics, runtime, memory, and release availability before choosing a backend; it does not require NumKong to copy OpenCV's API.

## Related backend work

- [StringZilla #302](https://github.com/ashvardanian/StringZilla/issues/302) — typed and per-channel LUT, including `uint8 -> float32`. StringZilla already owns the public LUT/`translate` path used by Albucore.

## Common API contract

These issues can land independently. The Python-facing APIs benefit from the same rules:

- Preserve arbitrary leading batch dimensions.
- Make spatial or reduction axes explicit.
- Support C=1, channel counts greater than four, and C>128 without reinterpreting the Tensor rank or shape.
- Accept NumPy buffers directly.
- Offer `out=` where aliasing is well-defined.
- Document dtype promotion, accumulation dtype, rounding, saturation, NaN behavior, and border behavior.
- Release the GIL around the complete native operation.
- Start with contiguous input when needed and state stride restrictions explicitly.
- Keep the native layer free of hidden thread ownership; callers can partition work when an API exposes useful row or batch ranges.

## Suggested implementation order for our workloads

1. Buffer-first cast, axis-aware statistics, mixed-dtype affine, comparisons/clamp/select, and RNG address the largest reusable numerical passes. StringZilla #302 covers the typed LUT path.
2. Resampling and separable filtering form the minimum native foundation for the new true 3D transform family.
3. Dense-weight `lerp`, direct matmul, histogram/bincount, channel gather, and elementary math cover frequent specialized passes.
4. Neighborhood filters, including mode filtering and true 3D morphology, extend the spatial family after the resampling/filtering foundation.

This issue is an index. Each linked issue contains its own minimum useful contract, correctness cases, and benchmark baselines.

Contributor guide

Open the contributing guide

Research direction

This issue is an index rather than a self-contained implementation task. Read the linked issues, especially #362 and #364, for the actual contracts, correctness cases, and benchmarks; completion means delivering one of those independently scoped kernels rather than changing this tracker.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, numpy, python
Domain
computer-vision, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.