boot-clj / boot-clj/boot

Watchable fileset reified in the REPL session

Open
#492 5 comments 0 reactions 0 assignees View on GitHub
Documentation
Dominant language
Clojure
Stars
1.7k
Forks
178
PR merge metrics
No merged PRs in 30d

Description

Filing this as an issue per discussions in Slack.

Once boot enters the repl task, no other task can run within **that** pipeline. If a user has a task or pipeline of tasks to run that should be triggered based on changes to the fileset, these can only be run within the same JVM by spawning a new boot pipeline from within the repl. In other words, the repl task blocks the pipeline, rendering a pipeline such as `boot watch a-reactive-task repl` inert because it only runs once through (only calling the `pre` of each task and never returning to the watch).

While the user can spawn a new boot pipeline from within the repl, if that pipeline involves a watch, a whole new set of file system watcher threads will be spawned. This scales very poorly as each directory watched, including every subdirectory within, requires a new thread (at least in OSX). At several hundred threads, most JVMs come to a crawl.

Boot itself syncs user directories using a set of watcher threads while the repl task is running. One possible solution to this problem overall, as discussed in Slack, is exposing the fileset within the repl and enabling the user to add watchers to that fileset that then can trigger and feed into tasks/pipelines. This would enable reuse of these filesystem watcher threads and decouple the watching of the filesystem from the triggering of pipeline(s).

Contributor guide

Open the contributing guide

Research direction

Start by tracing the REPL task, the watch task, and Boot pipeline execution, then locate how Boot maintains its fileset and filesystem watcher threads. Determine how the REPL could expose the existing fileset and how watchers would trigger tasks or pipelines without creating another watcher set. Done means the issue's pipeline scenario works while reusing the existing watcher threads.

Written by the indexing model from the issue text.

Assessment

Tech stack
clojure
Domain
build-system, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.