AnswerDotAI / AnswerDotAI/nbdev

[Feature Requests] Better integration with standard tooling

Aperta
#1,466 14 commenti 13 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
Jupyter Notebook
Stelle
5.3k
Fork
513
Merge medio
2g 30m
PR unite (30g)
8

Descrizione

> TL/DR: I'd like to make some changes to `nbdev` to integrate it with other tooling.

Been using extensively `nbdev` over the last couple of weeks, and have identified a few quirks that bug me. This issue is to collect them so that they don't get lost. If the devs are open to these features, I could even contribute some PRs myself.

Long story short, I have been trying hard to integrate `nbdev` with other tools that enable what are considered "best practices" in the community, namely I have been working on some repos using

- `uv` as a package and environment manager
- `ruff` for linting and formatting
- `mypy` for type checking (until `ruff` finished their work on the `red-knot` module!)
- `deptry` for dependency QA
- `nbQA` to be able to run some of these tools on notebooks
- And a few others that are not particularly relevant here

First hurdle I experienced was the duplication of settings data. Modern tooling relies on `pyproject.toml` for their settings, whereas `nbdev` uses `settings.ini` instead. Most of the `nbdev` settings get loaded in the `setup.py` file, which can be used together with `pyproject.toml` by defining the build-system as `setuptools`, but I'd rather use more modern alternatives like `hatchling`.

So, I tried moving as many settings as possible from `settings.ini` to `pyproject.toml`, and (much to my surprise) I found I could safely move almost all of them and still get the core functionality of `nbdev` working. Besides allowing for using modern tooling, this has the additional benefit of bootstrapping `nbdev` for new projects: with this setup `nbdev` can be declared to be a dev dependency instead of requiring to be installed at some outside place. This allows to run everything from a virtual environment created by `uv`. The thing that one would "lose" would be the capability of creating a new project with `nbdev_init` (but that can be sorted by using a package template to either clone or use with `uv init`).

The remining minimal set of options I need left in `settings.ini` to use `nbdev` is so barebones that I think there would be no big loss of functionality if all those settings were to be loaded from a `[tools.nbdev]` section in `pyproject.toml`.

I have a very early stages [simple project template](https://github.com/jlopezpena/python-template/) showcasing how I got my setup to work (still very much WIP as I keep finding quirks when using that setup with real projects).

So, here is a list of changes I'd like to see (and can potentially help to implement if core devs agree with the vision):

1. Allow `nbdev` to read settings from `pyproject.toml` and delegate the dependency management to other tools
2. Modify the `nbdev_export` command so that:
- It uses absolute exports instead of relative exports -- absolute imports are preferred in some setups for a variety of reasons, but my main pain point here is that changing absolute imports from my notebooks to relative ones in my library annoys `ruff` as it can change the import ordering
- It can use `ruff format` (or any other formatter) to prettify the generated source (instead of black)
3. Longer term - Allow for more flexible exports than just the library setup currently in effect. Eg being able to export to different "subpackages" for monorepo stuctures, or to be able to export a bunc of management scripts that are not part of the library code.

Does this resonate with the core devs vision for the project and the community needs? Any feedback either here or in my template much appreciated!

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Inizia esaminando settings.ini, setup.py e il comando nbdev_export, quindi confrontali con la configurazione [tools.nbdev] proposta in pyproject.toml. Definisci quali impostazioni possono essere spostate, come dovrebbero funzionare la gestione delle dipendenze e gli import assoluti e come verrebbe specificata la selezione del formatter. Done deve includere l’ambito concordato e test che coprano il comportamento di integrazione selezionato.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
jupyter, python
Ambito
build-system, developer-experience, tooling
Tipo di issue
Funzionalità
Difficoltà
5/5
Tempo stimato
Più di una settimana
Stato di attività
Ferma
Chiarezza
Da chiarire
Idoneità per principianti
25/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.