microsoft / microsoft/simplechat

Split Microsoft 365 actions with sharing approvals, live file RAG, and delegated workflow execution

Open
#1,493 1 comment 0 reactions 1 assignee View on GitHub

@paullizer is already working on this.

Since Sep 17, 2026.

enhancement P1 security_improvement
Dominant language
Python
Stars
152
Forks
116
Avg merge
7h 7m
Merged PRs (30d)
122

Description

## Summary

Replace new creation of the combined Microsoft Graph action with four independently configurable Microsoft 365 actions: **Calendar**, **Email**, **OneDrive**, and **SharePoint Online**. Add permission-aware live file RAG, source-level sharing acknowledgements integrated with Profile, chat, Approvals, and notifications, and explicitly authorized delegated Microsoft 365 execution for workflows.

This is a design-first feature. The implementation plan is being finalized; no application implementation is included in this issue creation.

## User Value

Bring Microsoft 365 information into conversations without copying SharePoint content into group workspaces that do not enforce individual source-file permissions. Users control disclosure of their information, action owners can limit acknowledgement duration, and background work can pause for the correct person's approval instead of using an application identity.

## Proposed Behavior

### Four action types and legacy compatibility

- Introduce Microsoft 365 Calendar, Microsoft 365 Email, Microsoft 365 OneDrive, and Microsoft 365 SharePoint Online, using shared delegated authentication and cloud-aware Graph infrastructure.
- Preserve existing Calendar/Email operations, delivery controls, agent references, and pending actions.
- Existing combined `msgraph` actions remain runnable and editable, but cannot be newly created, cloned, imported as new records, or recreated after deletion. Hide the combined type from all new-action catalogs and enforce the rule at server write boundaries.
- Apply the new sharing controls to existing legacy Email, Calendar, and OneDrive operations too.

### Conversation sharing and user preferences

- Support personal, shared personal, group-context single-user, and shared group conversations.
- Require sharing acknowledgement for shared personal/shared group conversations, not private conversations merely using group content. Apply acknowledgement when previously private Microsoft 365-containing history is shared later.
- Keep one source-level sharing preference each for Calendar, Email, OneDrive, and SPO in Profile/Settings, not a separate preference per action ownership scope.
- Offer **Allow this request**, **Allow for today** (until local midnight), **Always allow**, and **No**. New actions allow all three affirmative durations by default; authorized action editors can impose a shorter maximum.
- The shorter user/action lifetime wins. A permanent preference must not silently renew a day-limited grant; require another actual acknowledgement after expiry.
- Declining a source continues the request without that source and explains the limitation.
- Conversation sharing intentionally discloses resulting answers/citations to participants. This is not per-viewer Microsoft 365 ACL enforcement, and later source-access revocation cannot retract already published text.

### Unified approvals and audit

- Persist a user-owned approval request that can be answered inline in chat or through the existing Approvals UI, with notifications and links to the request.
- Record acknowledgement references and safe audit metadata in the conversation/request/workflow: user, source, decision, duration, timestamps, effective policy, and relevant run/step.
- Only the person whose delegated Microsoft 365 data is used can grant that acknowledgement. Administrative/group roles must not substitute for the data user's approval.
- Pause and resume the exact operation durably and idempotently; do not restart completed work or repeat mail/calendar side effects.

### Live OneDrive and SharePoint RAG

- Search all content the user is entitled to access within the selected source. **No action-level site/folder allowlists.** A user's request can narrow that particular search to a folder or file.
- SharePoint initially covers document-library files, not pages/list rows or on-premises SharePoint.
- Prefer **Microsoft 365 Copilot Retrieval API v1.0** for supported, Copilot-licensed users; use ordinary delegated Graph search and file-content retrieval for unsupported clouds and unlicensed users.
- Pay-as-you-go Retrieval is out of scope for this release. Do not enable billing or invoke metered retrieval for unlicensed users.
- Do not fall back on access denial, consent refusal, policy blocks, or empty results to circumvent restrictions. Both providers enforce the selected user's source permissions.
- Return grounded text and canonical source citations, with `SPO` source attribution. No File Sync source, scheduled crawl, workspace document copy, or Azure AI Search ingestion.
- Fast-analysis limits are soft approval thresholds. If useful analysis needs larger/more files, offer deeper analysis or a faster answer with explicit coverage limitations. Store separate OneDrive/SPO extended-analysis preferences and use the same approval/notification/audit mechanism. Process large files in stages within technical resource/model limits; do not silently treat truncated content as full-file analysis.

### Delegated workflows

- Add an explicit, revocable **Connect Microsoft 365 for workflows** control in Profile, backed by securely stored per-user delegated credentials and Disconnect/reconnect handling.
- Each Microsoft 365-using workflow must select an explicitly consenting **Run as** user. The binding applies to **both manual and scheduled workflow runs**; direct chat still uses the invoking participant.
- Clearly display the Run as account. Selection alone does not authorize its use; approval, workflow scope, source capabilities, current access, and credential state must be enforced server-side.
- Missing acknowledgement or sign-in pauses the operation and notifies the intended user through Approvals. Never fall back to a workflow owner, initiating user, or app-only token.

### Clouds

- Support Azure Commercial, Azure Government, and configured custom clouds from the first release.
- Current Microsoft documentation lists Copilot Retrieval for global service only, not Government L4/L5. Ordinary Graph Search and file download provide the Government path.
- Respect configured authority, Graph resource/API endpoints, and trusted SharePoint/download hosts. Distinguish Microsoft 365 GCC from GCC High/DoD; never silently redirect an unsupported cloud to Commercial.

## Acceptance Criteria

- [ ] Four new types are wired through discovery, schemas, governance, action/agent forms, runtime capability filtering, connection checks, and documentation.
- [ ] Existing combined actions and pending mail/calendar behavior remain usable; all new-creation and deleted-record restoration paths are rejected.
- [ ] Source-specific sharing preferences, action policy ceilings, local-midnight expiry, revocation, legacy enforcement, and sharing existing history work through authoritative server checks.
- [ ] Chat, Approvals, notifications, and conversation/workflow audit metadata refer to the same persisted decisions; concurrent/repeated decisions and resume operations are idempotent.
- [ ] A shared-conversation participant uses their own delegated account; workflow executions use only their approved Run as account, for both manual and scheduled runs.
- [ ] Workflow connection/disconnection, Run as consent, missing-token reauthentication, durable waiting/resume, and cancellation are implemented without app-only fallback or exposed credentials.
- [ ] Copilot Retrieval and Graph fallback produce source-cited RAG without workspace/index persistence; source access/protection failures cannot be bypassed.
- [ ] User-requested folder/file scope is respected without action-level restrictions; malformed filters cannot silently broaden the returned content.
- [ ] Larger/more-file analysis supports explicit deeper/faster choices, per-source preferences, progress/cancellation, and accurate coverage rather than silent truncation.
- [ ] Commercial, Government L4/L5, and custom-cloud endpoint/authority/provider behavior are covered; live qualification gaps are recorded honestly.
- [ ] Functional, route-policy, Azure Playwright, and documentation coverage are updated, including meaningful permission-denial, sharing, fallback, legacy, and workflow-resume cases.
- [ ] Application patch version and new test/feature-document version headers are updated when implemented; deployer version is incremented if deployer logic changes.

## Notes

- **Priority:** P1.
- **Size:** XL, due to cross-cutting identity, approval, workflow, provider, and UI work.
- **Assignee:** @paullizer.
- **Related:** Refs #954 (SharePoint sync) and #956 (Microsoft 365 external tab). This issue does not close those broader/separate capabilities.
- Main existing anchors: `functions_msgraph_operations.py`, `semantic_kernel_plugins/msgraph_plugin.py`, `functions_authentication.py`, `semantic_kernel_loader.py`, action save/validation routes, `functions_approvals.py`, `functions_notifications.py`, `functions_workflow_runner.py`, Profile, action/agent forms, and collaboration/chat execution.
- The existing generic approval implementation is administrative and disallows self-approval. The new user-data approval type needs distinct authorization without weakening administrative approvals.
- Existing workflow user contexts do not contain a durable delegated Microsoft 365 token cache; secure connection storage and explicit Run as authorization are necessary new work, not assumptions about existing support.

### Verified Microsoft references

- [Copilot Retrieval API: permissions, contract, cloud availability](https://learn.microsoft.com/en-us/microsoft-365/copilot/extensibility/api/ai-services/retrieval/copilotroot-retrieval)
- [Retrieval licensing, limitations, and filtering behavior](https://learn.microsoft.com/en-us/microsoft-365/copilot/extensibility/api/ai-services/retrieval/overview)
- [Graph Search: permission trimming and entity scopes](https://learn.microsoft.com/en-us/graph/api/resources/search-api-overview?view=graph-rest-1.0)
- [Graph Search cloud availability](https://learn.microsoft.com/en-us/graph/api/search-query?view=graph-rest-1.0)
- [Graph file-content download](https://learn.microsoft.com/en-us/graph/api/driveitem-get-content?view=graph-rest-1.0)
- [Microsoft Graph national-cloud endpoints](https://learn.microsoft.com/en-us/graph/deployments)

The final design will also specify secure token-cache storage, workflow revision/run-as invalidation, durable approval/resume states, and operational limits. These must not be approximated by frontend-only checks or broad exception fallbacks.
## Finalized working-memory and workflow design

The design discussion added these requirements to the scope above:

- **Shared working-memory foundation:** Reuse/extract the existing durable tabular blob-manifest and checkpoint patterns. Store captured evidence, intermediate results, notes, and resumable checkpoints in conversation-scoped files in the configured chat storage account. Do not rely on worker-local disk or a single model context window. Do not replace source evidence with lossy summaries.
- **Evidence sharing is intentional:** The sharing acknowledgement explicitly covers both answers and retained source evidence. Authorized conversation participants may reuse the published snapshot without their own Microsoft 365 source permissions. Private/unapproved staging remains inaccessible. New remote searches/downloads still require the direct-chat user or approved workflow Run as user's delegated access. Show capture time/version; do not imply stored evidence is a fresh source read. Revoking source access or disconnecting Microsoft 365 does not retract published copies.
- **Retention:** Working memory follows conversation retention, archival, restore, and deletion. Temporary downloads are removed when no longer needed. Manifests must support exact cleanup and safe fork/private-to-shared transitions; no cross-conversation/global cache or workspace index.
- **Adaptive analysis:** The initial fast windows are 3 content downloads per logical request, 25 MiB per file, and 12,000 file-context tokens, further bounded by the selected model's available context. These are approval thresholds, not silent full-file cutoffs. Approved deeper analysis proceeds in resumable, cancellable batches with page/slide/sheet/row coverage. Text, PDF, Word, PowerPoint, and spreadsheet/CSV formats are in scope; unsupported protected content, audio/video, and archives are not. Exact tabular calculations use captured rows, not summaries.
- **Saved workflow connections require Key Vault:** Protect a dedicated encryption key with Key Vault and store the per-account MSAL cache encrypted in a server-only connection store. Include account/tenant/cloud binding, concurrent refresh coordination, revocation fencing, key rotation, and no plaintext fallback. Normal interactive actions do not require Key Vault. Do not export/import credentials through ordinary application data tools.
- **Run as revision authorization:** Both manual and scheduled workflow runs use the selected consenting account. Material changes to instructions, referenced agent/action capabilities, run-as identity, runtime-input contract, or output destinations require that user's renewed authorization. Run-as credentials apply to Microsoft 365 calls, not unrelated SimpleChat privileges.
- **One approval record, multiple UIs:** Chat, Approvals, notifications, and conversation/run audit refer to the same conditional decision. Preserve the administrative approval rules for old request types. Only the subject user approves these data-use requests. Approval and execution are separate states; durable resume must not replay already-completed steps or ambiguous external mutations.

### Phased delivery

1. Contracts, delegated/cloud-aware transport, typed user-data approval handlers, and shared working-memory/checkpoint primitives.
2. Source-specific actions, legacy compatibility, Profile/action sharing policies, shared-conversation/history publication controls, and Approvals/notification/audit UI.
3. Copilot Retrieval plus Graph file RAG, supported-format extraction, adaptive staged analysis, retained evidence, source citations, and memory lifecycle.
4. Secure delegated workflow connections, explicit revision-bound Run as authorization, durable waiting/sign-in/resume, and scheduler/cancellation handling.
5. Cross-cloud qualification, regression and UI coverage, documentation/inventory, and controlled rollout. Block new combined Graph creation only when all replacement types are usable.

Broader adoption of the shared memory foundation by unrelated workflow and existing tabular analysis features is a **later follow-up**, not a full tabular rewrite in this issue.

### Qualification gates

- Verify licensed-user eligibility without calling Retrieval as a probe: an unlicensed request can be metered when a tenant independently enables PAYG. Unverified users use Graph; do not add tenant-wide license administrator permissions to normal chat users.
- Prove exact actor/cloud binding and durable pause/resume with real runtime modules and restart/concurrency scenarios, not only UI or substring checks.
- Validate available Graph/Copilot paths in target clouds; record unavailable live-tenant coverage rather than claiming mock coverage establishes deployed support.

Planning remains separate from implementation. No repository application changes have been made during this design work.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.