Vision Models

Vision models are language models that also accept image input. You send images alongside text in the same request, and the model reasons over both.

When to Use

Use a vision model when your prompt includes images:

• Describing, captioning, or classifying images
• Extracting text or data from screenshots and documents
• Visual question answering
• Comparing or reasoning across multiple images

Vision models still return text, not images. To generate images, see Image & Video.

AI SDK Method

Vision models use the same chatModel() method as language models. Pass image bytes or a URL as a file content part with an image/* media type:

import { readFile } from 'node:fs/promises';
import { createOLLM } from '@orgn/gateway';
import { generateText } from 'ai';

const ollm = createOLLM({ apiKey: process.env.OLLM_API_KEY });
const image = await readFile('photo.jpg');

const { text } = await generateText({
  model: ollm.chatModel('vercel_claude_sonnet_4_6'),
  messages: [{
    role: 'user',
    content: [
      { type: 'text', text: 'Describe this image.' },
      { type: 'file', data: image, mediaType: 'image/jpeg' },
    ],
  }],
});

Supported media types include image/jpeg, image/png, image/webp, image/gif, and any other image/* value the underlying model accepts. See the Vercel AI SDK integration for PDF and document input.

TEE Catalog

Vision models running in Trusted Execution Environments, on NEAR and Phala infrastructure with Intel TDX + NVIDIA H100 confidential compute.

ZDR Catalog

Vision-capable models running on Vercel's AI infrastructure with zero data retention provider agreements.