Doom CLI Rewrite

> [!IMPORTANT]
> This is a [roadmap](https://doomemacs.org/roadmap) issue and is used solely to announce and document progress (to make it easier for users to follow and maintainers to track). Please do not use it for discussion; use [our discussion board](https://doomemacs.org/discuss) instead.

I've been working on this CLI rewrite for a while. Over time, bug reports and feature requests have trickled in for things that this rewrite is already slated to address, directly or indirectly. To reduce the noise I'm consolidating those and futures posts into this one, to represent the rewrite:

## Primary goals of this rewrite

- [ ] (#8838) "consumption and development" workflow: allow users to modify package source directly without it causing conflicts with Doom's package manager.
+ [ ] (#4270) To make our CLI transactional (failures should revert back to where we started rather than be ignored).
+ [ ] (#2141) To offer a `doom rollback` command (i.e. keep track of "generations" of `doom sync`s).
+ [x] (#3490) Make `doom sync` smarter about `:recipe` changes and stale bytecode (so you don't need to do that `doom sync -u` and `doom build` dance).
+ [ ] Improve the doctor API and checks, so we get more useful checks _and_ the warnings/errors they emit are less cryptic.
+ (#2048) Check for misspelled modules and flags
+ (#4157) Warn users when they inadvertently use package.el
+ (#4190) Support for a private doctor.el in your DOOMDIR
+ (#2959) Perform more sanity checks on your private config
+ [ ] To fix our unit tests and add integration tests, powered by nix and buttercup.
+ [x] CI/CD, so bumping packages can be automated (and tested).
+ [ ] To handle straight.el's prompts better -- or prevent them entirely. Re-cloning packages is better UX than these confusing prompts.
+ [ ] Add a new `doom package ...` family of commands for per-package management (e.g. `doom package build PACKAGE` will spare you an expensive `doom build` to rebuild all packages).
+ [ ] Add a new `doom module ...` family of commands for toggling modules (or installing third party ones) programmatically (indirectly gives us a way to keep your init.el up-to-date across `doom upgrade`s).
+ [ ] Allow `doom run` to launch sandboxes and, if nix is available, do so from specific versions of Emacs.
+ [ ] (#4429, #4880) Make `doom install` more beginner friendly
+ [ ] (#4603) Add `-f` switch to `doom sync` to suppress prompts.
+ [ ] (#4608, #4823) Fix `package!`'s `:ignore` directive
+ [ ] (#4712) Mark packages to be skipped when updating
+ [ ] (#4816) Autoloads aren't reindexed if package recipe contains `:build (:not compile)`
+ [ ] (#5086) Build epdfinfo when pdf-tools package is installed/built.
+ [ ] (#5185) Isolate more in Doom's sandbox
+ [ ] (#5461) Have `doom sync -p` handle nativecomp artifacts.
+ [ ] (#5501) More informative/helpful output (e.g. that native compilation finished).
+ [ ] (#6687) First class support for literate configs (allows init.el, too, to be generated).

This is not an exhaustive list.

## Other bugs

+ [x] (#3844) Get /bin/sh on Windows to run chained scripts correctly
+ [x] (#3746) Fix `bin/doom` aborting unexpectedly when restarting itself
+ [ ] (#5362) Make `doom upgrade` atomic.
+ [ ] (#5590) Prompt less on `doom upgrade`.

-----

I'll keep this post updated until the rewrite is done. There is no ETA on this.