error: ⚠ Error: t.content.filter is not a function when using a Mod to attempt add vision capabilities

Summary

I made a mod to attempt to add Image vision capabilities to any model by detecting an image and then delegating to a cheap vision capably model then send that back into the original prompt flow.

But, when I used and tried within deepseek pro, I encountered the following error:

 describe what you see in this image [Image #1]

◼ [vision-bridge] Describing 1 image(s) via MiniMaxAI/MiniMax-M3…

⚠ Error: t.content.filter is not a function

  Type "continue" to try again. If the issue persists, contact support: https://commandcode.ai/discord
  Trace ID: 76ee1a3ecc68b48a9b9f04420863cebf

❯ continue

⚠ Error: t.content.filter is not a function

  Type "continue" to try again. If the issue persists, contact support: https://commandcode.ai/discord
  Trace ID: 00e918dd06ff0fdfd52ff1a662d19162

Here is the mod - not Sur if an issue with the mod, Command Code flow, or the vision model.

/**
 * Vision Bridge — describe/OCR images for models that lack vision.
 *
 * When a prompt includes an image but the current model can't see images,
 * this mod saves the image to a temp file and shells out to `cmd -p`
 * with a cheap vision model to describe it, then injects the description
 * in place of the image blocks.
 *
 * Cheap vision models (both available in Command Code):
 *   stepfun/step-3.7-flash     — $0.20/M in, $1.15/M out (fastest: 400 tok/s)
 *   MiniMaxAI/MiniMax-M3       — $0.30/M in, $1.20/M out (higher intel)
 *
 * Flags:
 *   vision-model     (string)  Model ID for vision (default: stepfun/step-3.7-flash)
 *   vision-prompt    (string)  Prompt sent to vision model for each image
 */

import type { ModApi } from "@commandcode/harness";
import { createHash } from "node:crypto";
import { writeFile, unlink } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";

const DEFAULT_MODEL = "MiniMaxAI/MiniMax-M3";
const FALLBACK_MODEL = "stepfun/step-3.7-flash";
const DEFAULT_PROMPT =
  "Describe this image in thorough detail. If it contains text, transcribe it verbatim (OCR). Include key visual elements, layout, colors, and context.";

interface ImageBlock {
  type: "image";
  source: { type: "base64"; media_type: string; data: string };
}

interface ContentBlock {
  type: string;
  text?: string;
  source?: unknown;
}

function hashData(data: string): string {
  return createHash("sha256").update(data).digest("hex").slice(0, 16);
}

function extForType(mediaType: string): string {
  if (mediaType.startsWith("image/")) return mediaType.slice(6);
  return "png";
}

export default function (cmd: ModApi): void {
  cmd.addFlag("vision-model", {
    type: "string",
    default: DEFAULT_MODEL,
    description: `Vision model ID (default: ${DEFAULT_MODEL})`,
  });
  cmd.addFlag("vision-prompt", {
    type: "string",
    default: DEFAULT_PROMPT,
    description: "Prompt sent to vision model for each image batch",
  });

  const cache = new Map<string, string>();

  async function describeImages(
    modelId: string,
    prompt: string,
    images: ImageBlock[],
    signal?: AbortSignal,
  ): Promise<string | null> {
    const tmpPaths: string[] = [];
    try {
      for (let i = 0; i < images.length; i++) {
        const img = images[i];
        const ext = extForType(img.source.media_type);
        const path = join(tmpdir(), `cmd-vision-${Date.now()}-${i}.${ext}`);
        await writeFile(path, Buffer.from(img.source.data, "base64"));
        tmpPaths.push(path);
      }

      const fileList = tmpPaths.join(" ");
      const fullPrompt = `${prompt}\n\nImage files: ${fileList}\n\nRead each image file above and describe its contents thoroughly. Output ONLY the descriptions, no extra commentary or introduction.`;

      const { stdout, stderr, code } = await cmd.exec({
        command: "cmd",
        args: [
          "-p",
          fullPrompt,
          "-m",
          modelId,
          "--max-turns",
          "3",
          "--no-session",
        ],
        signal,
      });

      if (code !== 0 && import.meta.env.DEV) {
        console.error(
          `[vision-bridge] cmd -p exited ${code}:`,
          stderr.slice(0, 300),
        );
      }

      return stdout?.trim() || null;
    } finally {
      for (const p of tmpPaths) {
        try {
          await unlink(p);
        } catch {
          // best effort
        }
      }
    }
  }

  cmd.hooks({
    transformContext: async ({ messages, signal }) => {
      let hasImages = false;
      for (const msg of messages) {
        if (
          msg.role === "user" &&
          Array.isArray((msg as { content: unknown }).content)
        ) {
          const blocks = (msg as { content: ContentBlock[] }).content;
          if (blocks.some((b) => b.type === "image")) {
            hasImages = true;
            break;
          }
        }
      }
      if (!hasImages) return messages;

      const model = (cmd.getFlag("vision-model") as string) || DEFAULT_MODEL;
      const visionPrompt =
        (cmd.getFlag("vision-prompt") as string) || DEFAULT_PROMPT;

      const hashToResult = new Map<string, string>();
      const uncached: ImageBlock[] = [];
      const batchHashes: string[] = [];

      for (const msg of messages) {
        if (
          msg.role !== "user" ||
          !Array.isArray((msg as { content: unknown }).content)
        )
          continue;
        for (const block of (msg as { content: ContentBlock[] }).content) {
          if (block.type !== "image") continue;
          const img = block as unknown as ImageBlock;
          const h = hashData(img.source.data);

          if (cache.has(h)) {
            hashToResult.set(h, cache.get(h)!);
          } else if (!hashToResult.has(h)) {
            uncached.push(img);
            batchHashes.push(h);
            hashToResult.set(h, "__PENDING__");
          }
        }
      }

      if (uncached.length > 0) {
        cmd.ui.notify(`Describing ${uncached.length} image(s) via ${model}…`);

        let desc: string | null = null;

        try {
          desc = await describeImages(model, visionPrompt, uncached, signal);
        } catch (err) {
          if (import.meta.env.DEV) {
            console.error("[vision-bridge] primary model failed:", String(err));
          }

          if (model !== FALLBACK_MODEL) {
            cmd.ui.notify(
              `Vision failed, trying ${FALLBACK_MODEL}…`,
              "warning",
            );
            try {
              desc = await describeImages(
                FALLBACK_MODEL,
                visionPrompt,
                uncached,
                signal,
              );
            } catch (err2) {
              if (import.meta.env.DEV) {
                console.error(
                  "[vision-bridge] fallback model failed:",
                  String(err2),
                );
              }
              cmd.ui.notify("Vision bridge: both models failed.", "warning");
            }
          } else {
            cmd.ui.notify("Vision bridge failed.", "warning");
          }
        }

        if (desc) {
          for (const h of batchHashes) {
            cache.set(h, desc);
            hashToResult.set(h, desc);
          }
        } else {
          for (const h of batchHashes) {
            hashToResult.delete(h);
          }
        }
      }

      if (hashToResult.size === 0) return messages;

      const out: Record<string, unknown>[] = [];
      for (const msg of messages) {
        if (
          msg.role !== "user" ||
          !Array.isArray((msg as { content: unknown }).content)
        ) {
          out.push(msg as Record<string, unknown>);
          continue;
        }

        const blocks = (msg as { content: ContentBlock[] }).content;
        if (!blocks.some((b) => b.type === "image")) {
          out.push(msg as Record<string, unknown>);
          continue;
        }

        const textParts: string[] = [];
        const descParts: string[] = [];

        for (const block of blocks) {
          if (block.type === "text" && block.text) {
            textParts.push(block.text);
          } else if (block.type === "image") {
            const img = block as unknown as ImageBlock;
            const h = hashData(img.source.data);
            const d = hashToResult.get(h);
            if (d) {
              descParts.push(
                `[Image description (${img.source.media_type}): ${d}]`,
              );
            }
          }
        }

        let finalText = "";
        if (textParts.length > 0) finalText += textParts.join("\n");
        if (descParts.length > 0) {
          if (finalText) finalText += "\n\n";
          finalText += descParts.join("\n\n");
        }

        if (finalText) {
          out.push({ role: "user", content: finalText });
        }
      }

      return out as typeof messages;
    },
  });
}

Expected Behavior

The mod to work ads described:

• detect image
• use smaller, cheap vision model to do vision
• send that back to main prompt model to be used.

Actual Behavior

Errors:

 describe what you see in this image [Image #1]

◼ [vision-bridge] Describing 1 image(s) via MiniMaxAI/MiniMax-M3…

⚠ Error: t.content.filter is not a function

  Type "continue" to try again. If the issue persists, contact support: https://commandcode.ai/discord
  Trace ID: 76ee1a3ecc68b48a9b9f04420863cebf

❯ continue

⚠ Error: t.content.filter is not a function

  Type "continue" to try again. If the issue persists, contact support: https://commandcode.ai/discord
  Trace ID: 00e918dd06ff0fdfd52ff1a662d19162

Steps to reproduce the issue

• create mod
• ask/prompt with an image in a model that does not support vision like deep seek pro

Command Code Version

1.4.2

Operating System

macOS

Terminal/IDE

ghostty

Shell

zsh

Additional context

No response

Beginne beim bereitgestellten transformContext-Hook und prüfe die von Command Code 1.4.2 erwartete Struktur des Nachrichteninhalts, insbesondere im Hinblick auf die zurückgegebene Benutzernachricht. Reproduziere das Problem mit DeepSeek Pro und einem Bild und verifiziere anschließend, dass die Bildbeschreibung den Haupt-Prompt erreicht, ohne den Fehler t.content.filter zu verursachen.