elisp-development-workflow.org

Elisp development workflow for dotfiles extras

The two-clone problem

Extras packages (org-extras, gptel-extras, etc.) live in two places:

• Working tree: ~/My Drive/dotfiles/emacs/extras/ — where all edits happen
• Elpaca clone: ~/.config/emacs-profiles/<profile>/elpaca/sources/dotfiles/emacs/extras/ — where Emacs loads from

Elpaca treats the dotfiles repo as a GitHub package (via use-personal-package, which generates an :ensure recipe pointing at github:benthamite/dotfiles). It clones the repo into elpaca/sources/dotfiles/ and byte-compiles from there into elpaca/builds/. The builds directory contains symlinks to the sources directory plus .elc files.

The working tree and the elpaca clone are separate git clones of the same repo. Edits go into the working tree; Emacs loads from the elpaca clone. They must be kept in sync.

Sync mechanism

Three git hooks in the working tree’s gitdir (/Users/pablostafforini/git-dirs/dotfiles/hooks/) propagate commits to the elpaca clone:

Hook	Trigger	Action
post-commit	Normal commits	Delegates to sync-elpaca-clone.sh
post-rewrite	Rebases, amends	Delegates to sync-elpaca-clone.sh
sync-elpaca-clone.sh	Called by the above	Fetches from the working tree’s gitdir into the elpaca clone, then reset --hard FETCH_HEAD

After the sync, sync-elpaca-clone.sh starts elpaca-rebuild-wait for every extras package whose .el file changed. The hook remains fire-and-forget to avoid deadlocking when Git is spawned synchronously by Emacs (for example, magit-commit-squash). Before it returns, it writes a pending state keyed by the committed revision and package. The bounded background helper changes that state to finished or failed and preserves its log under ~/.cache/dotfiles/elpaca-reload/COMMIT/. A later agent invocation of the same helper observes that exact post-commit state instead of scheduling a duplicate build.

The active elpaca profile is read from ~/.config/emacs-profiles/.current-profile (a cache file written by Emacs at startup), not from emacsclient, to avoid the same deadlock.

Key invariant: sync only propagates committed changes

The elpaca clone tracks git history, not the filesystem. Uncommitted edits in the working tree are invisible to the elpaca clone. This means:

• After git commit: elpaca clone is up to date, elpaca-rebuild compiles the new code.
• Before git commit: elpaca clone is stale, elpaca-rebuild compiles old code.

The load-elisp-after-edit.sh Claude Code hook fires on every Edit/Write of an .el file and schedules elpaca-extras-rebuild-and-reload. But if the edit hasn’t been committed yet, the rebuild compiles from the stale elpaca source. This is harmless because the post-commit hook triggers a second rebuild from the correct source. But it means that between editing and committing, the running Emacs may have stale code.

Batch testing (emacs --batch)

Use the source-aware batch runner:

~/My\ Drive/dotfiles/claude/bin/batch-test.sh YOUR-PACKAGE

The runner keeps Elpaca builds available for dependencies but loads the requested library by its canonical .el path. This prevents stale package bytecode from shadowing the edit. It fails on any stale-load warning and emits evidence bound to the repository, package, and current Elisp source revision. The commit guard recomputes that revision and rejects evidence from a different package or from source changed after the test.

Common pitfalls

Wrong elpaca profile

Multiple profiles may coexist (e.g. 8.1.0 and 8.1.0-dev). Always resolve the active profile dynamically:

emacsclient -e 'init-current-profile'

Never hardcode a profile name. Stale profiles have stale elpaca clones that are never synced.

elpaca-rebuild without elpaca-extras-reload

elpaca-rebuild byte-compiles the source into elpaca/builds/ but does not load the new code into the running Emacs session. elpaca-extras-reload is the step that calls load on each feature, which re-evaluates defconst, defvar, defun, etc.

In the live daemon, do not call elpaca-wait inside one long emacsclient expression or write an open-ended polling loop. After commit, use the bound live helper:

~/My\ Drive/dotfiles/claude/bin/elisp-live-verify PACKAGE -- '(PACKAGE-CHECK)'

Each emacsclient request has its own timeout, and the complete rebuild has a 120-second default deadline. The command succeeds only after finished and a successful package-named live expression. Its evidence names the repository, package, and commit. For a checkout whose repository name differs from its Elpaca ID, pass the repository name as PACKAGE and name the resolved Elpaca ID in the expression. For example, use emacs-slack and slack respectively.

Editing before committing

If you edit a file and immediately check whether Emacs picked up the change (e.g. evaluating a defconst), it won’t have — the edit hasn’t been committed, so the elpaca clone is stale. Commit first, then the post-commit hook handles sync + rebuild + reload automatically.

Full lifecycle of a change

1. Edit ~/My Drive/dotfiles/emacs/extras/foo-extras.el
2. load-elisp-after-edit.sh fires on a direct edit or on changed paths recovered from a composed Codex patch; for extras this pre-commit rebuild can still use the committed mirror
3. require-elisp-test-before-commit.sh blocks commit until source-aware batch evidence matches the repository, package, and source revision
4. Run batch-test.sh, which loads the canonical source path rather than package bytecode
5. git commit -> post-commit hook -> sync-elpaca-clone.sh: a. Fetches committed changes into elpaca clone b. reset --hard FETCH_HEAD c. Writes a pending state keyed by commit and package d. Starts bounded elpaca-rebuild-wait processing, which records finished or failed
6. Run elisp-live-verify foo-extras -- '(foo-extras-CHECK)'; it waits for that exact post-commit result and then exercises the named live path
7. Only its repository/package/commit-bound evidence clears that obligation; an unrelated live evaluation does not