finos / finos/morphir

Task graph with declared inputs, source tracking, and install as a graph node

Open
#788 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
209
Forks
65
Avg merge
8h 55m
Merged PRs (30d)
126

Description

## Context

The out-directory work (`docs/design/out-directory.md`) gives every task a scratch directory (`.dest`), a result record (`.json`) with an `inputs` provenance list, and an install step (`-o `) that copies a task's product into a user directory while keeping a ledger of the files it wrote. The record already names the single upstream edge (`resolve_ir_task`), and #785 covers hashing inputs so an unchanged task can be skipped.

What is still missing is the model Mill, Bazel, and Zig share: tasks declare their inputs (source files plus upstream task outputs), the tool hashes them, and every step, including install, is a node in one graph.

## What this issue covers

- **Declared sources.** A task states which source files it reads (today the compile task reads `[project].source_directory` implicitly). The set of sources that exist before a run is recorded on the result record alongside their hashes.
- **Input hashing and skip.** Builds on #785: a task whose declared sources and upstream results are unchanged is not rerun, and its `.dest` is not cleared. This makes the current clear-on-start behaviour safe by construction.
- **Install as a graph node.** The install step becomes its own task (`install/`), with its own record whose `inputs` name the task it installed from and whose `value` lists the files written into the user directory. That record is what today's `installed` ledger approximates; making it a node means it survives an upstream failure for free. The interim tombstone-record fix that keeps the ledger across failed runs can then be removed.
- **A task graph API.** A small in-process graph (nodes are task ids, edges are `inputs`) that `compile`, `transform/` (#786), `generate/`, and `install/` all register with, so ordering and skip decisions live in one place rather than in each command.

## Out of scope

The transform extension protocol itself (#786) and remote or cross-machine caching.

Contributor guide

Open the contributing guide

Research direction

Start by reading docs/design/out-directory.md and the existing result-record, install-ledger, and resolve_ir_task behavior described there. Review #785 and #786 for related hashing and transform work. Done means declared sources and upstream results drive skip decisions, compile/transform/generate/install register in one task graph, and install has its own record of inputs and written files.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
build-system, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.