Feature: File Upload Restrictions with Post-Processing Enforcement and Always-On Security Validation

Introduce two complementary features for file uploads in workspaces and chat:

1. Admin-configurable restrictions

   • Allow admins to define parameters (keywords, regex, or other rules) in the admin app settings to prevent certain files from being ingested.
   • Since file content may not always be visible before upload, the system should validate the content during or after chunking/extraction (e.g., using Document Intelligence or another extraction method).
   • If restricted content is detected, the system should roll back any data already processed, ensuring that nothing persists in AI Search or other services.
   • This feature should be toggleable in the admin app.

2. Always-on security validation

   • Independently of admin restrictions, all uploaded files should undergo validation to confirm they are what they claim to be (e.g., correct MIME type, file extension consistency, safe format).
   • This validation is mandatory and cannot be disabled.

Motivation

• Admin-configurable restrictions allow organizations to enforce governance and compliance controls dynamically. Example use cases include blocking sensitive content, disallowing certain file types, or enforcing naming conventions.
• Post-processing enforcement ensures that if content violations are detected only after chunking/extraction, no restricted data remains in the system.
• Always-on security validation prevents malicious or mislabeled files from entering the platform, ensuring baseline protection for all users.

Acceptance Criteria

• Admins can enable/disable the restriction feature in the admin app.
• Admins can provide restriction parameters (keywords, regex, file types, etc.).
• Restrictions are enforced during or after chunking/extraction.
• If violations are detected, all previously ingested chunks are removed from AI Search and other services.
• Users receive a clear error message when a file is blocked or rolled back.
• Security validation runs on all files, verifying type/format integrity.
• Security validation cannot be disabled.
• Logs or alerts are available for blocked or rolled-back uploads.

Additional Notes

• These features apply to workspace file uploads and chat file uploads.
• Security validation is always enforced, while admin restrictions are optional and configurable.
• The rollback mechanism is critical for cases where detection happens only after chunking.

No files, tests, or concrete entry points are named. Start by locating the admin-app settings and workspace/chat upload paths, then trace chunking or extraction through AI Search and the other services; done means configurable restrictions, mandatory file validation, rollback, user errors, and logging are covered by tests.