doom-initialize-packages loads package autoloads without resolving or ordering dependencies, crashing CLI commands
- Dominant language
- Emacs Lisp
- Stars
- 22.7k
- Forks
- 3.1k
- Avg merge
- 10h 46m
- Merged PRs (30d)
- 4
Description
## Summary
Every package-management CLI command (`doom gc`, `doom sync`, `doom build`, …) aborts with **exit 255** if any installed package hoists a bare macro-call form into its generated `-autoloads.el`, where that macro is defined by a (transitive) dependency.
`doom-initialize-packages` loads each package's straight autoloads but (1) never actually resolves dependencies, and (2) doesn't order them dependency-first — unlike `doom-profile--generate-loaddefs-packages`, which is correct. Interactive sessions are therefore unaffected; only the CLI crashes.
## Environment
- Doom: `2.2.0` @ `dce7f1d37`
- Emacs: `32.0.50` (also reproduces on 30/31)
- straight.el, batch (`bin/doom`) sessions
## Reproduce
Install any package whose generated autoloads contain a top-level macro call whose macro lives in a **transitive-only** dependency. A real-world example is [`isamert/lab.el`](https://github.com/isamert/lab.el): it `;;;###autoload`s an `(async-defun lab-list-project-pipelines …)` form, and `async-defun` is a macro from `async-await`, which `lab` pulls in only via `Package-Requires`. Emacs' autoload generator copies the whole form verbatim, so `lab-autoloads.el` contains a bare `(async-defun …)`.
```console
$ doom gc
x There was an unexpected runtime error
Message: Symbol's function definition is void
Details: (async-defun)
Backtrace:
(async-defun lab-list-project-pipelines (&optional project) …)
(load-with-code-conversion ".../build-32.0.50/lab/lab-autoloads.el" …)
(straight--load-package-autoloads "lab")
… doom-initialize-packages …
```
Exit code 255; the command's own logic never runs. The same crash hits `doom sync` and `doom build` (any caller of `doom-initialize-packages`).
## Root cause
The interactive and CLI paths build the package-autoload load list differently, and only the interactive one is correct.
**Interactive** — `doom-profile--generate-loaddefs-packages` (`lisp/lib/profiles.el`) walks the dependency tree **depth-first**, dependencies before dependents. Its own comment:
> Create a list of packages starting with the Nth-most dependencies by walking the package dependency tree depth-first. **This ensures any load-order constraints in package autoloads are always met.**
So the generated `90-loaddefs-packages.auto.el` emits `(autoload 'async-defun "async-await" …)` *before* the `(async-defun lab-… )` form — startup works.
**CLI** — `doom-initialize-packages` (`lisp/lib/packages.el`) does not:
```elisp
(cl-callf append packages (cons name (straight--get-dependencies name)))
…
(dolist (package (cl-delete-duplicates packages :test #'equal))
(straight-register-package package)
(let ((name (symbol-name package)))
(add-to-list 'load-path (directory-file-name (straight--build-dir name)))
(straight--load-package-autoloads name)))
```
Two defects:
1. **Dependency resolution is a no-op.** `name` is a **symbol** (from `doom-packages`), but `straight--get-dependencies` keys `straight--build-cache` by **string**:
```elisp
(defun straight--get-dependencies (package) ; "PACKAGE should be a string"
(nth 1 (gethash package straight--build-cache)))
```
Observed at runtime:
```
(straight--get-dependencies 'lab) ; => nil
(straight--get-dependencies "lab") ; => ("emacs" "request" "s" "f" "compat" "promise" "async-await")
```
So `(cons name (straight--get-dependencies name))` is always just `(list name)`. **Transitive-only dependencies' autoloads are never loaded at all** in CLI sessions — `async-await` is absent from the entire 331-package load list.
2. **No dependency-first ordering.** Even with no.1 fixed, `(cons name deps)` + keep-last `cl-delete-duplicates` does not guarantee a dependency's autoloads load before its dependents'.
Either defect alone breaks `lab`; together they guarantee the crash.
## Proposed fix
Load package autoloads dependency-first, mirroring `doom-profile--generate-loaddefs-packages`:
```diff
@@ -203,10 +203,17 @@ processed."
(cl-pushnew name doom-disabled-packages)
(when recipe
(straight-override-recipe (cons name recipe)))
- (cl-callf append packages (cons name (straight--get-dependencies name)))))))
- (dolist (package (cl-delete-duplicates packages :test #'equal))
- (straight-register-package package)
- (let ((name (symbol-name package)))
+ (push (symbol-name name) packages)))))
+ ;; Load autoloads dependency-first, so load-order constraints hoisted into
+ ;; package autoloads are met (mirrors `doom-profile--generate-loaddefs-packages').
+ (let (ordered)
+ (cl-labels ((walk (pkg)
+ (when (and (stringp pkg) (not (member pkg ordered)))
+ (mapc #'walk (straight--get-dependencies pkg))
+ (push pkg ordered))))
+ (mapc #'walk (nreverse packages)))
+ (dolist (name (nreverse ordered))
+ (straight-register-package (intern name))
(add-to-list 'load-path (directory-file-name (straight--build-dir name)))
(straight--load-package-autoloads name))))))
```
(The `walk` is the same shape as the one already in `doom-profile--generate-loaddefs-packages`; it could be factored into a shared helper.)
## Validation
With the patch applied (and no other workaround), the autoload load order becomes dependency-first:
```
… promise → iter2 → async-await → lab …
```
and `doom gc` / `doom sync` / `doom build` run to completion (exit 0, no `void-function`). Verified on Emacs 32.0.50, Doom 2.2.0 @ dce7f1d37.
## Workaround (no core change)
Ensure the dependency is loaded before the dependent's autoloads. E.g. a module `cli.el` that advises `straight--load-package-autoloads` to `(require 'async-await)` right before `lab` is loaded. This only patches one package; the fix above addresses the class of bug.
Contributor guide
Research direction
Start in lisp/lib/packages.el at doom-initialize-packages and compare its autoload loading with doom-profile--generate-loaddefs-packages in lisp/lib/profiles.el. Verify dependency names and dependency-first ordering using the lab/async-await reproduction, then run doom gc, doom sync, and doom build; done means all complete with exit 0 and no void-function error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- emacs-lisp
- Domain
- cli, tooling
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 75/100