google / google/minimalloc

Adding a Python API

Open
#14 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
193
Forks
17
PR merge metrics
No merged PRs in 30d

Description

Hi @mmoffitt,

Thanks for your excellent work on this allocator. Reading your paper was a lot of fun!

I wanted to experiment with your allocator, particularly to embed it in a larger compiler flow and benchmark it against other allocators I've been using. For this, I needed a Python interface (as many ML compiler workflows are Python-based), so I went ahead and created one using pybind11. You can find the initial version [here](https://github.com/fpedd/minimalloc/commit/5a79bb6df28f9877648a79f9a58289cae938531a). While it's still somewhat rough, I've ported all the tests to pytest and added examples and a Python CLI.

With this, one can now:
```
pip install minimalloc
```
and use it like this:
```python
import minimalloc as mm
problem = mm.from_csv_file("benchmarks/examples/input.12.csv")
problem.capacity = 12
solver = mm.Solver()
solution = solver.solve(problem)
print(solution) # Solution(offsets=[8, 8, 4, 4, 0])
```
Since the Python interface is just a wrapper, the performance is identical to the C++ interface. I've added a small [script](https://github.com/fpedd/minimalloc/tree/fpedd/python_bindings/scripts/run-challenging-compare.sh) that uses both the C++ and Python CLIs to verify identical results and comparable performance:
```bash
(venv) minimalloc$ bash scripts/run-challenging-compare.sh
benchmarks/challenging/A.1048576.csv [Python] Elapsed time: 4.011s [C++] Elapsed time: 4.046s PASS
benchmarks/challenging/B.1048576.csv [Python] Elapsed time: 2.791s [C++] Elapsed time: 2.776s PASS
benchmarks/challenging/C.1048576.csv [Python] Elapsed time: 0.528s [C++] Elapsed time: 0.540s PASS
benchmarks/challenging/D.1048576.csv [Python] Elapsed time: 2.769s [C++] Elapsed time: 2.757s PASS
benchmarks/challenging/E.1048576.csv [Python] Elapsed time: 2.849s [C++] Elapsed time: 2.859s PASS
benchmarks/challenging/F.1048576.csv [Python] Elapsed time: 4.609s [C++] Elapsed time: 4.641s PASS
benchmarks/challenging/G.1048576.csv [Python] Elapsed time: 0.977s [C++] Elapsed time: 0.975s PASS
benchmarks/challenging/H.1048576.csv [Python] Elapsed time: 1.407s [C++] Elapsed time: 1.421s PASS
benchmarks/challenging/I.1048576.csv [Python] Elapsed time: 3.026s [C++] Elapsed time: 3.031s PASS
benchmarks/challenging/J.1048576.csv [Python] Elapsed time: 1.087s [C++] Elapsed time: 1.093s PASS
benchmarks/challenging/K.1048576.csv [Python] Elapsed time: 0.425s [C++] Elapsed time: 0.430s PASS
All tests passed - Python and C++ APIs produced identical results
```
I wanted to ask whether you'd be interested in upstreaming this to your project. If so, I'd be happy to clean up the PR, split my changes into smaller, more reviewable PRs, and help maintain the Python bindings going forward.

This would enable many opportunities down the line, such as publishing to PyPI, making your work easily accessible to a much wider audience for integration into their compilers and experiments. For example, an ETH Zurich project called [Deeploy](https://github.com/pulp-platform/Deeploy) has integrated your allocator in their [Python](https://github.com/pulp-platform/Deeploy/blob/2fa3c319882e2c6fccdd73d4217ac2f72b30b3d3/Deeploy/TilingExtension/TilerExtension.py#L231) code. But they currently have to go through CSV files.

Thanks again for your work and for making it available to the community.

Best,
Fabian

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.