pylint-dev / pylint-dev/astroid

[discussion] Use a rust crate to make astroid faster, or part of astroid faster

Open
#2,014 16 comments 0 reactions 0 assignees View on GitHub
Discussion 🤔 topic-performance
Dominant language
Python
Stars
582
Forks
357
Avg merge
1d 1h
Merged PRs (30d)
23

Description

I was reading [this article](https://www.peterbaumgartner.com/blog/wrapping-a-rust-crate-in-a-python-package/) lately. With pydantic, ruff and other python package using rust to provide faster python using binding offered by https://github.com/PyO3/pyo3 . I've been wondering if we should explore the possibility to do that too.

Potential contributor to astroid are scarce, so we rejected using cython earlier (In https://github.com/PyCQA/astroid/pull/606#issuecomment-498015223 and elsewhere with hippo91 but I can't find the link right now).

However:
- rust is well loved and has been the most loved language for a few year on stackoverflow. So we might find contributors after all ?
- ``ruff`` performance proves it's possible as a linter and also that it's a massive boost (10x or more) and a game changer
- astroid's code is convoluted, maybe a way to avoid painfully typing it and maintaining it ? Daniel wanted to make it internal, well, porting to rust is a way to do whatever we like ? 😄

Problem I foresee:
- I don't know how astroid works, it's not a given that we're going to architecture a new package better ?
- We rely on the ast module right now, I don't know if we can rely on the ast from rust. The fact that ruff do not handle pattern matching seems to indicate that no we can't.

Maybe it's possible to use rust on the bottleneck only, i.e. on inference.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.