AI setup

VitNode integrates the Vercel AI SDK. Register models in vitnode.api.config.ts once, then use them inside any API handler via c.get("ai").

Quick startCopy Anchor Link

1. Pick a providerCopy Anchor Link

A model is either a provider instance from an AI SDK provider package or an AI Gateway id string such as 'openai/gpt-4o-mini'.

• Provider package. Install it in your API app and set its key, for example OPENAI_API_KEY for @ai-sdk/openai.

bun i @ai-sdk/openai

• AI Gateway. Install nothing. The AI SDK sends id strings through the Vercel AI Gateway, which reads AI_GATEWAY_API_KEY from your .env file.

2. Configure models in the API configCopy Anchor Link

import { openai } from '@ai-sdk/openai'
import { buildApiConfig } from '@vitnode/core/vitnode.config'

export const vitNodeApiConfig = buildApiConfig({
  ai: {
    models: [
      {
        id: 'default',
        name: 'GPT-4o Mini',
        model: openai('gpt-4o-mini'), // or the gateway id 'openai/gpt-4o-mini'
        capabilities: ['text', 'structured-output', 'streaming'],
        pricing: {
          rates: { inputPerMillion: '0.15', outputPerMillion: '0.60' },
        },
      },
    ],
  },
})

capabilities and pricing are optional, but AI actions rely on them. Capabilities decide which model can run an action, and pricing turns token usage into money. See Model capabilities and Costs and budgets.

3. Use it in a route handlerCopy Anchor Link

For features people can trigger, run a registered AI action. It checks permissions, limits and budgets, and records the cost:

handler: async (c) => {
  const { output } = await c.get('ai').run({
    action: SUMMARIZE_TOPIC_AI_ACTION,
    input: c.req.valid('json'),
  })

  return c.json({ text: output }, 200)
}

SUMMARIZE_TOPIC_AI_ACTION is the action key you create in AI actions. For internal tooling, you can also resolve the raw model and call the SDK directly. See Direct SDK calls.

Model capabilitiesCopy Anchor Link

A capability says what a configured language model can do. VitNode never guesses it from the model's name. A model without capabilities is treated as ["text"].

Every AI action declares the capabilities it needs. When a run starts, VitNode picks the first configured model that has all of them. If no model fits, the run fails with AI_MODEL_INCOMPATIBLE before anything is sent to a provider.

Model pricingCopy Anchor Link

pricing is what the model costs on this connection, in USD per one million tokens. Prices are decimal strings, so money never goes through floating point:

{
  id: 'default',
  name: 'Claude Sonnet',
  model: anthropic('claude-sonnet-4-5'),
  capabilities: ['text', 'image-input', 'structured-output', 'streaming'],
  pricing: {
    rates: {
      inputPerMillion: '3',
      outputPerMillion: '15',
      cacheReadPerMillion: '0.30',
      cacheWritePerMillion: '3.75',
    },
  },
}

Prices live only in this config. The AdminCP has no price editor, so a price change is a code change you deploy. A malformed price stops the server at boot.

Without pricing, a call's cost stays unknown, never zero. Under a budget cap, a run on a model with no pricing fails with AI_PRICING_MISSING. Costs and budgets covers tiers, flat fees and gateway vs direct pricing.

Model optionsCopy Anchor Link

ai also accepts providerAdapters: readers for providers that report their own cost. The AI Gateway and OpenRouter readers are built in. See Provider-reported cost.

Multiple models & embeddingsCopy Anchor Link

Configure dedicated models for fast completions, reasoning, and vector embeddings:

ai: {
  models: [
    {
      id: 'default',
      name: 'GPT-4o Mini',
      model: openai('gpt-4o-mini'),
      capabilities: ['text', 'structured-output', 'streaming'],
    },
    {
      id: 'reasoning',
      name: 'Claude Sonnet',
      model: 'anthropic/claude-sonnet-5', // AI Gateway id
      capabilities: ['text', 'image-input', 'structured-output', 'streaming'],
    },
  ],
  embeddingModels: [
    { id: 'default', name: 'Text Embedding 3', model: openai.embedding('text-embedding-3-small') },
  ],
}

Resolve specific models by ID:

const smartModel = c.get('ai').model('reasoning')
const embedModel = c.get('ai').embeddingModel()
const capabilities = c.get('ai').capabilities('reasoning') // ['text', 'image-input', ...]

embeddingModel() throws when no embeddingModels are configured, and imageModel() when no imageModels are. Admins can assign a different model (and a fallback) to each action in the AdminCP. Core refuses an assigned model that lacks a required capability.

Reading the model list in the browserCopy Anchor Link

Provider instances, API keys and prices stay on the server. c.get("ai").models() returns only the public half of each entry:

The middleware route publishes this list, and the browser reads it with useMiddlewareConfigQuery():

import { useMiddlewareConfigQuery } from '@vitnode/core/tanstack/auth'

export const ModelPicker = () => {
  const { data } = useMiddlewareConfigQuery()

  if (data.ai.models.length === 0) return <p>AI is not configured.</p>

  return (
    <select className="w-full">
      {data.ai.models.map((model) => (
        <option key={model.id} value={model.id}>
          {model.name}
        </option>
      ))}
    </select>
  )
}

The first entry is the default, which c.get("ai").model() resolves when a handler gets no id. The list is empty when no ai.models are configured, so a picker can hide itself instead of offering nothing.

Learn moreCopy Anchor Link