[FEATURE] Fail gracefully on attachments the provider will not accept

Scope check

• This is core LLM communication (not application logic)
• This benefits most users (not just my use case)
• This can't be solved in application code with current RubyLLM
• I read the Contributing Guide

Due diligence

• I searched existing issues
• I checked the documentation

What problem does this solve?

One attachment a provider will not accept fails the whole request. A single .pptx in an otherwise valid prompt means the question never reaches the model, and a chat app shows an error instead of an answer. It would be more useful for the model to say it could not read the file.

RubyLLM knows whether a provider accepts a file, but it only says so by raising, and there is no general way to ask beforehand.

An application cannot fill the gap, because it has to decide before a provider is chosen. It ends up reading a protocol constant and hardcoding the rest of the support table. Model metadata does not help either: the input modalities for Sonnet 5 are text, image, and PDF, which says nothing about .xlsx against .pptx.

#737 and #826 added support for more file types. This issue is about what happens when a file is still unsupported, and about letting an application decide.

Current workaround

On 1.16.0 I subclass Attachment and convert anything Bedrock would reject, at construction.

class PromptAttachment < RubyLLM::Attachment
  ACCEPTED_TYPES = %i[image pdf text].freeze
  ACCEPTED_DOCUMENT_FORMATS = RubyLLM::Providers::Bedrock::Media::SUPPORTED_DOCUMENT_FORMATS

  def initialize(source, filename: nil)
    super
    extract unless accepted?
  end

  private

  def accepted?
    ACCEPTED_TYPES.include?(type) || (type == :document && ACCEPTED_DOCUMENT_FORMATS.include?(extension))
  end

  def extract
    @source = StringIO.new(TextExtraction.call(self) || unsupported_note)
    @filename = "#{filename}.txt"
    @content = nil
    determine_mime_type
  end
end

TextExtraction dispatches to a per-format adapter and returns nil when no adapter claims the file or it will not parse, so anything unreadable falls through to a note naming the file and its type. It works, but it's not an elegant solution:

• It decides before a provider is known, so every provider gets Bedrock's answer.
• It reads a provider constant that has since moved to Protocols::Converse.
• Clearing @content is easy to miss. Extraction reads it, so a stale memo puts the original binary inside the text attachment.
• Overriding type or mime_type instead of swapping the source recurses, because type reads mime_type. Reassigning @source and re-running a private method is the only way through.

Proposed solution

1. Make support introspectable. Give each protocol media module a supported_attachment? predicate and have the existing dispatch consult it, so the raise reads the predicate instead of repeating it. Converse already does this for its document formats, to decide auto-uploads. Add Protocol#renderable_attachment?, which combines protocol support with model modality. This is useful on its own.

Input modalities are accurate for image, audio, video, and PDF, but there is no document member (the vocabulary's file is OpenRouter passthrough), and Sonnet 5 on Bedrock reads .xlsx natively. So document attachments skip the modality check and use the protocol's answer.

2. Make the failure mode configurable. An unsupported_attachment option taking :raise or :text, defaulting to :raise so current behavior is unchanged, with a matching Chat#with_unsupported_attachments and an Agent macro. Background jobs and user-facing chats want different answers.

I left out a :drop mode. A file that disappears without the model being told is worse than a note saying it was unsupported.

3. Allow converters to be registered. A RubyLLM.register_attachment_converter hook taking a block that receives the attachment and returns a String, or nil to decline. Blocks run in registration order. An exception is logged and treated as declining. Format parsers stay in application code. With nothing registered, :text still works and yields a note naming the file and its type.

4. Substitute at render time. Protocol#preprocess_message already replaces attachments per request for auto-upload. Add a branch there for unsupported ones. The provider and model are both in scope, and the message is rebuilt rather than mutated, so stored history keeps the original file and a chat moved to another provider is checked against that provider.

Two things I would leave out of a first pass. A scanned document has no text to extract, and there a note is worse than handing the binary to a vision model. The default note's wording also deserves its own discussion, since it reaches the model as user-role content.

Why this belongs in RubyLLM

The library has this information and the application does not. At render time we know the provider and the model, and the support table is there.

The workaround above answers a provider-specific question before a provider exists, and two of its four problems are private implementation details of Attachment that no application should be holding.

The format parsers stay in application code or separate gems. This keeps Ruby LLM out of the text extraction business but lets users standardize on gems or roll their own.

Start with Protocol#preprocess_message and the existing Converse attachment-support dispatch, then inspect Attachment and the Chat#with_unsupported_attachments and Agent configuration entry points. The work is done when provider- and model-aware support checks, configurable :raise/:text handling, and registered converter callbacks preserve the default behavior while substituting unsupported attachments at render time.