doomemacs / doomemacs/core

doom-initialize-packages loads package autoloads without resolving or ordering dependencies, crashing CLI commands

Open
#8,825 0 comments 1 reaction 0 assignees View on GitHub
core re:packages
Dominant language
Emacs Lisp
Stars
22.7k
Forks
3.1k
Avg merge
10h 46m
Merged PRs (30d)
4

Description

## Summary

Every package-management CLI command (`doom gc`, `doom sync`, `doom build`, …) aborts with **exit 255** if any installed package hoists a bare macro-call form into its generated `-autoloads.el`, where that macro is defined by a (transitive) dependency.

`doom-initialize-packages` loads each package's straight autoloads but (1) never actually resolves dependencies, and (2) doesn't order them dependency-first — unlike `doom-profile--generate-loaddefs-packages`, which is correct. Interactive sessions are therefore unaffected; only the CLI crashes.

## Environment

- Doom: `2.2.0` @ `dce7f1d37`
- Emacs: `32.0.50` (also reproduces on 30/31)
- straight.el, batch (`bin/doom`) sessions

## Reproduce

Install any package whose generated autoloads contain a top-level macro call whose macro lives in a **transitive-only** dependency. A real-world example is [`isamert/lab.el`](https://github.com/isamert/lab.el): it `;;;###autoload`s an `(async-defun lab-list-project-pipelines …)` form, and `async-defun` is a macro from `async-await`, which `lab` pulls in only via `Package-Requires`. Emacs' autoload generator copies the whole form verbatim, so `lab-autoloads.el` contains a bare `(async-defun …)`.

```console
$ doom gc
x There was an unexpected runtime error
Message: Symbol's function definition is void
Details: (async-defun)
Backtrace:
(async-defun lab-list-project-pipelines (&optional project) …)
(load-with-code-conversion ".../build-32.0.50/lab/lab-autoloads.el" …)
(straight--load-package-autoloads "lab")
… doom-initialize-packages …
```

Exit code 255; the command's own logic never runs. The same crash hits `doom sync` and `doom build` (any caller of `doom-initialize-packages`).

## Root cause

The interactive and CLI paths build the package-autoload load list differently, and only the interactive one is correct.

**Interactive** — `doom-profile--generate-loaddefs-packages` (`lisp/lib/profiles.el`) walks the dependency tree **depth-first**, dependencies before dependents. Its own comment:

> Create a list of packages starting with the Nth-most dependencies by walking the package dependency tree depth-first. **This ensures any load-order constraints in package autoloads are always met.**

So the generated `90-loaddefs-packages.auto.el` emits `(autoload 'async-defun "async-await" …)` *before* the `(async-defun lab-… )` form — startup works.

**CLI** — `doom-initialize-packages` (`lisp/lib/packages.el`) does not:

```elisp
(cl-callf append packages (cons name (straight--get-dependencies name)))

(dolist (package (cl-delete-duplicates packages :test #'equal))
(straight-register-package package)
(let ((name (symbol-name package)))
(add-to-list 'load-path (directory-file-name (straight--build-dir name)))
(straight--load-package-autoloads name)))
```

Two defects:

1. **Dependency resolution is a no-op.** `name` is a **symbol** (from `doom-packages`), but `straight--get-dependencies` keys `straight--build-cache` by **string**:
```elisp
(defun straight--get-dependencies (package) ; "PACKAGE should be a string"
(nth 1 (gethash package straight--build-cache)))
```
Observed at runtime:
```
(straight--get-dependencies 'lab) ; => nil
(straight--get-dependencies "lab") ; => ("emacs" "request" "s" "f" "compat" "promise" "async-await")
```
So `(cons name (straight--get-dependencies name))` is always just `(list name)`. **Transitive-only dependencies' autoloads are never loaded at all** in CLI sessions — `async-await` is absent from the entire 331-package load list.

2. **No dependency-first ordering.** Even with no.1 fixed, `(cons name deps)` + keep-last `cl-delete-duplicates` does not guarantee a dependency's autoloads load before its dependents'.

Either defect alone breaks `lab`; together they guarantee the crash.

## Proposed fix

Load package autoloads dependency-first, mirroring `doom-profile--generate-loaddefs-packages`:

```diff
@@ -203,10 +203,17 @@ processed."
(cl-pushnew name doom-disabled-packages)
(when recipe
(straight-override-recipe (cons name recipe)))
- (cl-callf append packages (cons name (straight--get-dependencies name)))))))
- (dolist (package (cl-delete-duplicates packages :test #'equal))
- (straight-register-package package)
- (let ((name (symbol-name package)))
+ (push (symbol-name name) packages)))))
+ ;; Load autoloads dependency-first, so load-order constraints hoisted into
+ ;; package autoloads are met (mirrors `doom-profile--generate-loaddefs-packages').
+ (let (ordered)
+ (cl-labels ((walk (pkg)
+ (when (and (stringp pkg) (not (member pkg ordered)))
+ (mapc #'walk (straight--get-dependencies pkg))
+ (push pkg ordered))))
+ (mapc #'walk (nreverse packages)))
+ (dolist (name (nreverse ordered))
+ (straight-register-package (intern name))
(add-to-list 'load-path (directory-file-name (straight--build-dir name)))
(straight--load-package-autoloads name))))))
```

(The `walk` is the same shape as the one already in `doom-profile--generate-loaddefs-packages`; it could be factored into a shared helper.)

## Validation

With the patch applied (and no other workaround), the autoload load order becomes dependency-first:

```
… promise → iter2 → async-await → lab …
```

and `doom gc` / `doom sync` / `doom build` run to completion (exit 0, no `void-function`). Verified on Emacs 32.0.50, Doom 2.2.0 @ dce7f1d37.

## Workaround (no core change)

Ensure the dependency is loaded before the dependent's autoloads. E.g. a module `cli.el` that advises `straight--load-package-autoloads` to `(require 'async-await)` right before `lab` is loaded. This only patches one package; the fix above addresses the class of bug.

Contributor guide

Open the contributing guide

Research direction

Start in lisp/lib/packages.el at doom-initialize-packages and compare its autoload loading with doom-profile--generate-loaddefs-packages in lisp/lib/profiles.el. Verify dependency names and dependency-first ordering using the lab/async-await reproduction, then run doom gc, doom sync, and doom build; done means all complete with exit 0 and no void-function error.

Written by the indexing model from the issue text.

Assessment

Tech stack
emacs-lisp
Domain
cli, tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
75/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.