HamzaHassanain / HamzaHassanain/polyman

Implement Smart Compilation Cache with Dependency Tracking

Open
#11 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
35
Forks
3
PR merge metrics
No merged PRs in 30d

Description

Implement an intelligent caching system for compiled executables that tracks source file changes and dependencies. Skip compilation when source hasn't changed, dramatically speeding up repeated runs and enabling smart verification that only re-runs affected components.

Problem

Currently:

1. Every polyman verify recompiles everything (slow)
2. No dependency tracking (changing generator doesn't re-generate tests)
3. No incremental verification
4. Wastes time on unchanged components

**Proposed Solution**

1. Compilation Cache:
```
.polyman/cache/
├── compiled/
│ ├── solutions/
│ │ ├── main.exe (cached binary)
│ │ └── main.meta.json (metadata: hash, timestamp, flags)
│ ├── generators/
│ │ └── gen.exe
│ ├── validator/
│ │ └── val.exe
│ └── checker/
│ └── chk.exe
└── dependency-graph.json
```

2. Smart Cache Invalidation:
```
interface CacheMetadata {
sourceFile: string;
sourceHash: string; // SHA-256 of source
compiledBinary: string;
compiledAt: Date;
compileFlags: string[];
dependencies: string[]; // e.g., ["testlib.h"]
dependencyHashes: Record;
}

async function needsRecompilation(source: string): Promise {
const meta = readCacheMetadata(source);
if (!meta) return true; // No cache

// Check if source changed
const currentHash = await hashFile(source);
if (currentHash !== meta.sourceHash) return true;

// Check if dependencies changed
for (const [dep, oldHash] of Object.entries(meta.dependencyHashes)) {
const currentDepHash = await hashFile(dep);
if (currentDepHash !== oldHash) return true;
}

// Check if binary still exists
if (!fs.existsSync(meta.compiledBinary)) return true;

return false; // Cache valid!
}
```
3. Dependency Graph:
```
{
"nodes": {
"solutions/main.cpp": {
"type": "solution",
"dependencies": ["testlib.h"],
"affects": ["test-outputs"]
},
"generators/gen.cpp": {
"type": "generator",
"dependencies": ["testlib.h"],
"affects": ["testsets/tests/test3.txt", "testsets/tests/test4.txt"]
},
"validator/val.cpp": {
"type": "validator",
"dependencies": ["testlib.h"],
"affects": ["validation-results"]
}
},
"edges": [
{ "from": "testlib.h", "to": "solutions/main.cpp" },
{ "from": "generators/gen.cpp", "to": "testsets/tests/test3.txt" },
{ "from": "testsets/tests/test3.txt", "to": "test-outputs" }
]
}
```

**Cache Invalidation Scenarios**

Automatic invalidation:

- Source file modified
- Dependency (e.g., testlib.h) modified
- Compilation flags changed in Config.json
- Cached binary deleted/corrupted
- Cache format version mismatch

User-triggered:

-
- polyman cache clear
- polyman verify --no-cache
- Manual file deletion in .polyman/cache/

Success Criteria

- Compilation skipped when source unchanged (>10x faster)
- Dependency tracking works for testlib.h and local includes
- Cache invalidation is reliable (no stale binaries)
- --smart verification only re-runs affected steps
- Cache survives across sessions
- Cache size management (auto-cleanup old entries)
- Works cross-platform (Windows/Linux/macOS)
- Clear cache status and management commands
- Shows time saved in verification output
- Performance Impact

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.