Performance: Handle large vendor folder changes without freezing (e.g., Golang vendor updates)

## Performance Issue

The application needs to handle very large diffs, particularly vendor folder updates in Golang projects, without freezing or requiring the user to kill the application. GitHub itself struggles with these, and we should handle them better.

## Problem Description

When updating vendor folders in Golang projects (or similar large dependency updates in other languages), the diff can contain thousands of files with millions of lines of changes. This causes:
- UI freezing/hanging
- Excessive memory usage
- Browser tab crashes
- Need to force-quit the application

## Example Case

**Real-world example:** https://github.com/openfaas/ingress-operator/pull/66
- Golang vendor folder update
- Thousands of files changed
- GitHub's web UI struggles to render this
- Often requires killing the browser tab

## Current Behavior

- Application attempts to render entire diff at once
- No pagination or virtualization for large diffs
- No warning about large diff size
- No option to skip rendering large diffs
- Memory usage grows unbounded
- UI becomes unresponsive

## Expected Behavior

- Application remains responsive even with massive diffs
- Smart handling of vendor/dependency folders
- Options to view large diffs in a performant way
- Warnings and alternatives for extremely large changes

## Proposed Solution

### 1. Diff Size Detection and Warnings

```typescript
interface DiffAnalysis {
totalFiles: number;
totalLines: number;
estimatedRenderTime: number;
memoryRequired: number;
hasVendorChanges: boolean;
hasLargeBinaryFiles: boolean;
}

class DiffAnalyzer {
async analyzeDiff(pr: PullRequest): Promise {
const analysis = {
totalFiles: pr.changed_files,
totalLines: 0,
estimatedRenderTime: 0,
memoryRequired: 0,
hasVendorChanges: false,
hasLargeBinaryFiles: false
};

// Check for vendor/node_modules/dependencies
const vendorPatterns = [
/^vendor\//,
/^node_modules\//,
/^packages\//,
/^third_party\//,
/^deps\//,
/^dependencies\//,
/go\.sum$/,
/package-lock\.json$/,
/yarn\.lock$/,
/Gemfile\.lock$/,
/Cargo\.lock$/
];

for (const file of pr.files) {
analysis.totalLines += file.additions + file.deletions;

if (vendorPatterns.some(pattern => pattern.test(file.filename))) {
analysis.hasVendorChanges = true;
}

if (file.additions + file.deletions > 10000) {
analysis.hasLargeBinaryFiles = true;
}
}

// Estimate rendering performance
analysis.estimatedRenderTime = this.estimateRenderTime(analysis);
analysis.memoryRequired = this.estimateMemoryUsage(analysis);

return analysis;
}

getSizeCategory(analysis: DiffAnalysis): 'small' | 'medium' | 'large' | 'massive' {
if (analysis.totalLines < 500) return 'small';
if (analysis.totalLines < 5000) return 'medium';
if (analysis.totalLines < 50000) return 'large';
return 'massive';
}
}
```

### 2. Smart Diff Rendering Strategies

```typescript
class SmartDiffRenderer {
async renderDiff(pr: PullRequest, analysis: DiffAnalysis) {
const category = this.getSizeCategory(analysis);

switch (category) {
case 'small':
case 'medium':
return this.renderFullDiff(pr);

case 'large':
return this.renderProgressiveDiff(pr);

case 'massive':
return this.renderSummaryView(pr, analysis);
}
}

renderSummaryView(pr: PullRequest, analysis: DiffAnalysis) {
return (

### 3. Virtual Scrolling for Large Diffs

```typescript
class VirtualizedDiffViewer {
private visibleRange = { start: 0, end: 50 };
private fileBuffer = 10; // Files to render outside viewport

async renderVirtualized(files: DiffFile[]) {
return (

{({ index, style }) => (

### 4. Progressive Loading

```typescript
class ProgressiveDiffLoader {
private loadedFiles = new Set();
private loading = false;
private observer: IntersectionObserver;

async loadDiffProgressively(pr: PullRequest) {
const files = await this.getFileList(pr);

// Group files by importance
const groups = {
sourceCode: files.filter(f => this.isSourceCode(f)),
tests: files.filter(f => this.isTest(f)),
config: files.filter(f => this.isConfig(f)),
vendor: files.filter(f => this.isVendor(f)),
other: files.filter(f => this.isOther(f))
};

// Load in priority order
await this.loadGroup(groups.sourceCode, 'Source Code');
await this.loadGroup(groups.tests, 'Tests');
await this.loadGroup(groups.config, 'Configuration');

// Vendor files loaded on demand only
this.setupLazyLoading(groups.vendor);
}

private async loadGroup(files: DiffFile[], groupName: string) {
const BATCH_SIZE = 20;

for (let i = 0; i < files.length; i += BATCH_SIZE) {
const batch = files.slice(i, i + BATCH_SIZE);

// Show progress
this.updateProgress({
group: groupName,
loaded: i,
total: files.length
});

// Load batch
await this.loadBatch(batch);

// Allow UI to breathe
await this.yieldToUI();
}
}

private yieldToUI(): Promise {
return new Promise(resolve => {
requestIdleCallback(() => resolve(), { timeout: 100 });
});
}
}
```

### 5. Memory Management

```typescript
class DiffMemoryManager {
private maxMemoryMB = 500;
private currentUsage = 0;
private diffCache = new LRUCache();

async renderWithMemoryLimit(file: DiffFile): Promise {
const estimatedSize = this.estimateMemoryUsage(file);

// If single file is too large, render summary only
if (estimatedSize > this.maxMemoryMB * 0.5) {
return this.renderSummaryOnly(file);
}

// Free memory if needed
while (this.currentUsage + estimatedSize > this.maxMemoryMB) {
this.evictOldestDiff();
}

const rendered = await this.renderDiff(file);
this.diffCache.set(file.path, rendered);
this.currentUsage += estimatedSize;

return rendered;
}

private evictOldestDiff() {
const oldest = this.diffCache.pop();
if (oldest) {
this.currentUsage -= this.getSize(oldest);
// Clear from DOM
this.clearFromDOM(oldest);
}
}
}
```

### 6. Vendor Folder Handling

```typescript
class VendorFolderHandler {
private vendorPatterns = [
'vendor/**',
'node_modules/**',
'third_party/**',
'**/*.lock',
'**/go.sum',
'**/go.mod'
];

async handleVendorChanges(pr: PullRequest) {
const vendorFiles = pr.files.filter(f =>
this.vendorPatterns.some(p => minimatch(f.path, p))
);

if (vendorFiles.length > 100) {
return {
type: 'summary',
component: ,
expandable: true,
lazyLoad: true
};
}

return {
type: 'normal',
files: vendorFiles
};
}
}

function VendorChangesSummary({ files }) {
const stats = calculateVendorStats(files);

return (

### 7. Performance Monitoring

```typescript
class DiffPerformanceMonitor {
private metrics = {
renderStartTime: 0,
renderEndTime: 0,
memoryBefore: 0,
memoryAfter: 0,
filesRendered: 0,
framesDropped: 0
};

async monitorRendering(renderFn: () => Promise) {
this.metrics.renderStartTime = performance.now();
this.metrics.memoryBefore = performance.memory?.usedJSHeapSize || 0;

// Monitor frame drops
let lastFrameTime = performance.now();
const checkFrames = setInterval(() => {
const now = performance.now();
if (now - lastFrameTime > 100) { // More than 100ms = dropped frames
this.metrics.framesDropped++;
}
lastFrameTime = now;
}, 16);

try {
await renderFn();
} finally {
clearInterval(checkFrames);
this.metrics.renderEndTime = performance.now();
this.metrics.memoryAfter = performance.memory?.usedJSHeapSize || 0;

this.reportMetrics();
}
}

private reportMetrics() {
const duration = this.metrics.renderEndTime - this.metrics.renderStartTime;
const memoryUsed = this.metrics.memoryAfter - this.metrics.memoryBefore;

if (duration > 3000 || memoryUsed > 100 * 1024 * 1024) {
console.warn('Poor diff rendering performance detected', {
duration: `${duration}ms`,
memory: `${(memoryUsed / 1024 / 1024).toFixed(2)}MB`,
framesDropped: this.metrics.framesDropped
});

// Show user warning
this.showPerformanceWarning();
}
}
}
```

## Acceptance Criteria

- [ ] Large vendor folder updates don't freeze the UI
- [ ] Warning shown for diffs over threshold (e.g., 10k lines)
- [ ] Option to view file list only without diffs
- [ ] Option to exclude vendor files from view
- [ ] Virtual scrolling implemented for large file lists
- [ ] Progressive loading of diff content
- [ ] Memory usage stays under reasonable limit (e.g., 500MB)
- [ ] "Download diff" option for local viewing
- [ ] Graceful degradation for massive diffs
- [ ] Performance metrics tracked and reported
- [ ] Test with actual example: https://github.com/openfaas/ingress-operator/pull/66
- [ ] No browser tab crashes
- [ ] Cancel/abort option for diff loading
- [ ] Clear feedback on loading progress

## Testing

1. Load https://github.com/openfaas/ingress-operator/pull/66
2. Verify UI remains responsive
3. Verify memory usage stays reasonable
4. Test different viewing options
5. Verify no need to kill the application
6. Test with other large vendor updates (node_modules, Rust cargo, Python venv)

## Success Metrics

- Render time for 10k+ file diff: < 5 seconds
- Memory usage for large diff: < 500MB
- UI remains responsive (< 100ms input lag)
- No browser crashes
- User can view essential changes without viewing vendor files

🤖 Generated with [Claude Code](https://claude.ai/code)

The issue names no repository files, tests, or diff-rendering entry point; begin by locating the current PR diff loading and rendering path. Define a bounded milestone for keeping massive diffs responsive, then identify tests or measurements that demonstrate the UI no longer freezes while preserving a usable way to inspect the changes.

Written by the indexing model from the issue text.