Proposal: optional native parser backend with pure-Python fallback

python-dotenv optional native parser backend

Problem

python-dotenv parsing is on the configuration path of real consumers such as
pydantic-settings. We measured complete consumer calls, not parser-only
throughput, on the same macOS arm64 / CPython 3.14 environment.

Proposal

• Keep the current pure-Python parser and pip install python-dotenv default.
• Add an optional native extra whose backend-only dependency is
  fast-dotenv-rs-backend.
• Keep the Rust package separate from the dotenv namespace and console
  script; the adapter only converts lossless parser records into the existing
  Binding and Original objects.
• If the extra/backend is absent, the existing pure-Python path remains in
  use. If an explicitly selected backend violates its versioned contract or
  raises, surface that failure rather than silently masking it.

Runtime evidence

• Upstream base: a00cb2eed0704cd6d2071b2004c37e95ccc86ee5.
• Backend implementation: fast-dotenv-rs commits
  7cbdca0 and 95ea5c1 in the Master integration clone; shared Rust core
  and backend-only wheel are ready for review.
• Upstream adapter patch: python-dotenv commits
  0c1d17b9878ae2b390616fbd8b64f605f9fd9d67 and
  f5856435e229485040e0e11530a191b0f51007df.
• Exact Binding differential: 1,088 records, including 500 generated valid
  and 500 generated malformed records; native OFF, native ON, and backend
  direct paths each retained 0 mismatches.
• Upstream suite: 264 passed, 1 skipped in both OFF and ON runs, using a
  portable fixture for the host-specific printenv --version test.
• Native call proof: fast_dotenv_rs_backend.parse_bindings was observed on
  the same parse_stream path; backend and official package coexist without
  a top-level dotenv collision.
• Real consumer benchmark: dotenv_values, load_dotenv,
  pydantic-settings service/worker settings, repeated loads, and cold
  startup across small/medium/large fixtures. Semantic hashes matched;
  30 p50 metrics classified as 21 win / 8 neutral / 1 loss under a 2%
  neutral band. The only loss was small cold CLI startup (-1.6675 ms);
  representative warm savings were 0.1266 ms for small dotenv_values,
  0.7477 ms for medium, and 4.9392 ms for large.

Maintainer questions

1. Is an optional backend extra acceptable, with backend wheels maintained
   outside the default python-dotenv release path?
2. Should activation use the native extra, another name, or a different
   boundary?
3. Which platform, PyPy/free-threaded, supply-chain, and rollback guarantees
   are required before merge?

This is a concrete proposal with a merge-ready local patch. The backend is not
yet published to PyPI, and the current wheel evidence is macOS arm64 only;
those are intentionally open release gates, not hidden claims.

Commencez par le chemin parse_stream existant et examinez comment les enregistrements du parser deviennent des objets Binding et Original ; examinez ensuite les commits d’adaptateur cités et le point d’entrée fast_dotenv_rs_backend.parse_bindings. Exécutez la suite upstream dans les modes native OFF et ON et comparez les résultats différentiels. La tâche ne sera considérée comme terminée qu’après les décisions des maintainers concernant extra, la prise en charge des plateformes et des interpréteurs, les garanties de la chaîne d’approvisionnement et le rollback avant la release.