Watchable fileset reified in the REPL session
- Dominant language
- Clojure
- Stars
- 1.7k
- Forks
- 178
- PR merge metrics
- No merged PRs in 30d
Description
Filing this as an issue per discussions in Slack.
Once boot enters the repl task, no other task can run within **that** pipeline. If a user has a task or pipeline of tasks to run that should be triggered based on changes to the fileset, these can only be run within the same JVM by spawning a new boot pipeline from within the repl. In other words, the repl task blocks the pipeline, rendering a pipeline such as `boot watch a-reactive-task repl` inert because it only runs once through (only calling the `pre` of each task and never returning to the watch).
While the user can spawn a new boot pipeline from within the repl, if that pipeline involves a watch, a whole new set of file system watcher threads will be spawned. This scales very poorly as each directory watched, including every subdirectory within, requires a new thread (at least in OSX). At several hundred threads, most JVMs come to a crawl.
Boot itself syncs user directories using a set of watcher threads while the repl task is running. One possible solution to this problem overall, as discussed in Slack, is exposing the fileset within the repl and enabling the user to add watchers to that fileset that then can trigger and feed into tasks/pipelines. This would enable reuse of these filesystem watcher threads and decouple the watching of the filesystem from the triggering of pipeline(s).
Contributor guide
Research direction
Start by tracing the REPL task, the watch task, and Boot pipeline execution, then locate how Boot maintains its fileset and filesystem watcher threads. Determine how the REPL could expose the existing fileset and how watchers would trigger tasks or pipelines without creating another watcher set. Done means the issue's pipeline scenario works while reusing the existing watcher threads.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- clojure
- Domain
- build-system, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100