Context Annotations and Context Groups for Copilot Chat Attachments

# Feature Request: Context Annotations and Context Groups for Copilot Chat Attachments

## Summary

Introduce **Context Annotations** and **Context Groups** in GitHub Copilot Chat for VS Code, allowing users to associate structured notes, explanations, assumptions, intent, and guidance directly with attached files, symbols, folders, and code selections.

Today, attachments are treated as passive context objects. Users must describe the relationship between attached context and their request within the main prompt, forcing both the user and the LLM to infer attachment-to-instruction mappings.

Adding attachment-specific annotations and grouping capabilities would make complex multi-file conversations significantly easier to manage and improve context clarity for both users and AI models.

---

## Motivation

VS Code Copilot Chat already provides multiple mechanisms for attaching context, including:

- Files
- Folders
- Symbols
- Code selections
- Drag-and-drop context
- `#` references and mentions
- Codebase references

These mechanisms help users provide richer context to Copilot. However, attached resources currently lack a way to carry their own explanatory metadata or intent.

References:

- VS Code Context Management: https://code.visualstudio.com/docs/chat/copilot-chat-context
- VS Code Chat Overview: https://code.visualstudio.com/docs/chat/chat-overview
- Visual Studio Copilot Context References: https://learn.microsoft.com/en-us/visualstudio/ide/copilot-chat-context-references

---

## Problem Statement

When selecting a range of code and adding it to Copilot Chat, the selection is added as context only.

The same behavior applies when:

- Attaching multiple files
- Attaching folders
- Adding code selections
- Dragging files into chat
- Referencing multiple symbols

Users often need to communicate additional information about each attached context, such as:

- Why it is relevant
- Expected behavior
- Constraints
- Assumptions
- Problem locations
- Architectural roles
- Desired outcomes

Today there is no structured way to associate that information directly with the attached context object.

Instead, users must embed these explanations inside the prompt itself.

---

## Example of Current Workflow

### Attached Context

- AuthService.ts
- TokenValidator.ts
- ApiGateway.ts (selected range)

### User Intent

- AuthService.ts contains the legacy implementation.
- TokenValidator.ts contains the desired behavior.
- The selected ApiGateway.ts range contains the suspected bug.

### Current Prompt

```text
AuthService.ts contains the old implementation.

TokenValidator.ts contains the expected behavior.

The selected range in ApiGateway.ts is where the bug occurs.

Can you propose a migration strategy?
```

### Problem

Copilot must:

1. Process the attached context.
2. Process the natural language explanation.
3. Infer which explanation belongs to which attachment.
4. Build those relationships internally.
5. Only then focus on the actual request.

As the number of attachments grows, this becomes increasingly difficult to manage.

---

# Proposed Solution

Introduce a structured attachment metadata system.

Each attached context item should optionally support:

- Annotation
- Description
- Intent
- Notes
- Constraints
- Comments

directly attached to the context object.

---

## 1. Context Annotations

### File-Level Annotation

Instead of:

```text
AuthService.ts
TokenValidator.ts
```

Allow:

```text
AuthService.ts
Note: Legacy authentication implementation

TokenValidator.ts
Note: Desired target implementation
```

---

### Symbol-Level Annotation

```text
AuthenticateUser()
Note: Existing logic to preserve

ValidateToken()
Note: Source of truth implementation
```

---

### Code Selection Annotation

```text
ApiGateway.ts L120-L168
Note: Suspected bug location
```

or

```text
PaymentProcessor.cs L50-L95
Note: Current implementation fails under concurrency
```

This allows users to supply context-specific rationale directly at the point of attachment.

---

## 2. Context Groups

Users should be able to organize related attachments into logical groups.

### Example

```text
Authentication Flow

AuthService.ts
TokenValidator.ts
LoginController.ts

Shared Note:
Represents the desired architecture.
```

Another group:

```text
Problem Area

ApiGateway.ts L120-L168

Shared Note:
Reproduction starts here.
```

This removes the need for repeatedly describing relationships in the main prompt.

---

# Suggested UI

A possible UX pattern could be inspired by browser tab grouping.

Users would create context groups and optionally attach notes to each item.

---

## Expanded View

```text
╭──────────────────────────────────────────╮
│ Authentication Flow │
│──────────────────────────────────────────│
│ AuthService.ts │
│ TokenValidator.ts │
│ LoginController.ts │
│ │
│ Note: Desired architecture │
╰──────────────────────────────────────────╯

╭──────────────────────────────────────────╮
│ Problem Area │
│──────────────────────────────────────────│
│ ApiGateway.ts L120-L168 │
│ │
│ Note: Suspected bug location │
╰──────────────────────────────────────────╯
```

---

## Collapsed View

```text
▶ Authentication Flow (3)
▶ Problem Area (1)
```

---

# Workflow Comparison

## Current

```mermaid
flowchart TD

A[Attach File A]
B[Attach File B]
C[Attach Code Range]

D[Write Mapping Explanations in Prompt]

E[LLM Infers Relationships]

F[Interpret Actual Request]

A --> D
B --> D
C --> D

D --> E
E --> F
```

---

## Proposed

```mermaid
flowchart TD

A[Attach Context]
B[Add Context Annotation]
C[Create Context Groups]

D[Structured Attachment Metadata]

E[Interpret Actual Request]

A --> D
B --> D
C --> D

D --> E
```

---

# ASCII Illustration

## Current State

```text
+--------------------------------+
| Attachments |
+--------------------------------+
| AuthService.ts |
| TokenValidator.ts |
| ApiGateway.ts L120-L168 |
+--------------------------------+

Prompt:

"AuthService is old.
TokenValidator is new.
ApiGateway contains the bug."

Copilot must infer:
Which explanation belongs
to which attachment.
```

---

## Proposed State

```text
+--------------------------------+
| AuthService.ts |
| Note: Legacy implementation |
+--------------------------------+

+--------------------------------+
| TokenValidator.ts |
| Note: Expected implementation |
+--------------------------------+

+--------------------------------+
| ApiGateway.ts L120-L168 |
| Note: Suspected bug location |
+--------------------------------+

Prompt:

"Propose a migration strategy."
```

The attachment relationship becomes explicit and machine-readable.

---

# Benefits

## Better Prompt Structure

Intent is colocated with the relevant context rather than mixed into the request.

---

## Reduced Ambiguity

Attachment relationships become explicit instead of inferred.

---

## Better Multi-File Reasoning

Especially useful for:

- Refactoring
- Architecture reviews
- Root cause analysis
- Debugging
- Security investigations
- Migration planning
- Large-scale code changes

---

## Improved User Experience

Users can naturally explain:

- Why a file matters
- What a code selection represents
- Which implementation is preferred
- Which area contains the issue

without polluting the actual prompt.

---

## Potential Token Efficiency

Instead of repeatedly explaining attachment relationships inside every prompt, that information becomes structured metadata available alongside the attachment.

---

## Better Scalability

As Copilot continues to support larger context windows and more sophisticated codebase interactions, structured context becomes increasingly important.

Context annotations provide a natural evolution of the existing context attachment model rather than introducing an entirely new workflow.

---

# Alternative Names

Potential naming options:

- Context Annotations
- Attachment Notes
- Context Metadata
- Context Comments
- Context Labels
- Annotated Context
- Context Grouping

---

# Example User Scenarios

## Scenario 1: Bug Investigation

```text
Group: Reproduction Path

OrderController.cs
Note: Entry point

PaymentService.cs
Note: Incorrect state transition

PaymentRepository.cs
Note: Data inconsistency found here
```

Prompt:

```text
Identify the root cause.
```

---

## Scenario 2: Architecture Review

```text
Group: Existing Design

CurrentAuthService.ts
LegacyTokenService.ts

Note:
Current production implementation
```

```text
Group: Desired Design

NewIdentityProvider.ts

Note:
Target architecture
```

Prompt:

```text
Suggest a migration plan.
```

---

## Scenario 3: Refactoring

```text
CustomerController.cs
Note:
Keep public API unchanged.

CustomerService.cs
Note:
Safe to redesign internally.
```

Prompt:

```text
Refactor for maintainability.
```

---

# Conclusion

Copilot Chat already provides powerful mechanisms for attaching contextual information. However, attached context remains passive and cannot naturally carry user intent.

Adding **Context Annotations** and **Context Groups** would:

- Reduce ambiguity
- Improve prompt clarity
- Improve multi-file reasoning
- Scale better for complex engineering workflows
- Enable more structured AI interactions

This enhancement would build directly on existing VS Code context workflows while significantly improving usability for real-world software engineering scenarios.

---

## Related Issues

- #283137: Pin attached context files in Copilot Chat
- Focuses on persistence of attached context across chat turns.
- This proposal focuses on annotation and grouping of attached context.

- #261979: Group Conversations with Git Branch Context
- Focuses on organizing conversation threads.
- This proposal focuses on organizing attached context and associated intent.