AnswerDotAI / AnswerDotAI/nbdev

[Feature Requests] Better integration with standard tooling

Open
#1,466 14 comments 13 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
5.3k
Forks
513
Avg merge
2d 30m
Merged PRs (30d)
8

Description

> TL/DR: I'd like to make some changes to `nbdev` to integrate it with other tooling.

Been using extensively `nbdev` over the last couple of weeks, and have identified a few quirks that bug me. This issue is to collect them so that they don't get lost. If the devs are open to these features, I could even contribute some PRs myself.

Long story short, I have been trying hard to integrate `nbdev` with other tools that enable what are considered "best practices" in the community, namely I have been working on some repos using

- `uv` as a package and environment manager
- `ruff` for linting and formatting
- `mypy` for type checking (until `ruff` finished their work on the `red-knot` module!)
- `deptry` for dependency QA
- `nbQA` to be able to run some of these tools on notebooks
- And a few others that are not particularly relevant here

First hurdle I experienced was the duplication of settings data. Modern tooling relies on `pyproject.toml` for their settings, whereas `nbdev` uses `settings.ini` instead. Most of the `nbdev` settings get loaded in the `setup.py` file, which can be used together with `pyproject.toml` by defining the build-system as `setuptools`, but I'd rather use more modern alternatives like `hatchling`.

So, I tried moving as many settings as possible from `settings.ini` to `pyproject.toml`, and (much to my surprise) I found I could safely move almost all of them and still get the core functionality of `nbdev` working. Besides allowing for using modern tooling, this has the additional benefit of bootstrapping `nbdev` for new projects: with this setup `nbdev` can be declared to be a dev dependency instead of requiring to be installed at some outside place. This allows to run everything from a virtual environment created by `uv`. The thing that one would "lose" would be the capability of creating a new project with `nbdev_init` (but that can be sorted by using a package template to either clone or use with `uv init`).

The remining minimal set of options I need left in `settings.ini` to use `nbdev` is so barebones that I think there would be no big loss of functionality if all those settings were to be loaded from a `[tools.nbdev]` section in `pyproject.toml`.

I have a very early stages [simple project template](https://github.com/jlopezpena/python-template/) showcasing how I got my setup to work (still very much WIP as I keep finding quirks when using that setup with real projects).

So, here is a list of changes I'd like to see (and can potentially help to implement if core devs agree with the vision):

1. Allow `nbdev` to read settings from `pyproject.toml` and delegate the dependency management to other tools
2. Modify the `nbdev_export` command so that:
- It uses absolute exports instead of relative exports -- absolute imports are preferred in some setups for a variety of reasons, but my main pain point here is that changing absolute imports from my notebooks to relative ones in my library annoys `ruff` as it can change the import ordering
- It can use `ruff format` (or any other formatter) to prettify the generated source (instead of black)
3. Longer term - Allow for more flexible exports than just the library setup currently in effect. Eg being able to export to different "subpackages" for monorepo stuctures, or to be able to export a bunc of management scripts that are not part of the library code.

Does this resonate with the core devs vision for the project and the community needs? Any feedback either here or in my template much appreciated!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.