AI Setup

VitNode integrates the Vercel AI SDK. Register models in vitnode.api.config.ts once, then use them inside any API handler via c.get("ai").

Quick startCopy Anchor Link

1. Configure models in the API configCopy Anchor Link

A model can be an AI Gateway id string or a direct provider instance:

import { openai } from '@ai-sdk/openai'
import { buildApiConfig } from '@vitnode/core/vitnode.config'

export const vitNodeApiConfig = buildApiConfig({
  ai: {
    models: [
      {
        id: 'default',
        name: 'GPT-4o Mini',
        model: openai('gpt-4o-mini'),
        capabilities: ['text', 'structured-output', 'streaming'],
        pricing: {
          rates: { inputPerMillion: '0.15', outputPerMillion: '0.60' },
        },
      },
    ],
  },
})

Set OPENAI_API_KEY in your .env file.

capabilities and pricing are optional, but AI actions rely on them: capabilities decide which model can run an action, and pricing turns token usage into money. See Model capabilities and Costs, Points & Budgets.

2. Use it in a route handlerCopy Anchor Link

For user-facing features, run a registered AI action. It checks permissions, limits and budgets, and records the cost:

handler: async (c) => {
  const { output } = await c.get('ai').run({
    action: EXCERPT_AI_ACTION,
    input: c.req.valid('json'),
  })

  return c.json({ text: output })
}

For internal tooling, you can also resolve the raw model and call the SDK directly. See AI Usage.

Model capabilitiesCopy Anchor Link

A capability says what a configured language model can do. VitNode never guesses it from the model's name. A model without capabilities is treated as ["text"].

Every AI action declares the capabilities it needs. When a run starts, VitNode picks the first configured model that has all of them. If no model fits, the run fails with AI_MODEL_INCOMPATIBLE before anything is sent to a provider.

Model pricingCopy Anchor Link

pricing is what the model costs on this connection, in USD per one million tokens. Prices are decimal strings, so money never goes through floating point:

{
  id: 'default',
  name: 'Claude Sonnet',
  model: anthropic('claude-sonnet-4-5'),
  capabilities: ['text', 'image-input', 'structured-output', 'streaming'],
  pricing: {
    rates: {
      inputPerMillion: '3',
      outputPerMillion: '15',
      cacheReadPerMillion: '0.30',
      cacheWritePerMillion: '3.75',
    },
  },
}

Prices live only in this config. The AdminCP has no price editor, so a price change is a code change you deploy. A malformed price stops the server at boot.

Without pricing, a call's cost stays unknown, never zero. Under a budget cap, a model with no pricing is refused with AI_PRICING_MISSING. Tiers, flat fees and gateway vs direct pricing are covered in Costs, Points & Budgets.

Model optionsCopy Anchor Link

ai also accepts providerAdapters: readers for providers that report their own cost. The AI Gateway and OpenRouter readers are built in. See Provider-reported cost.

Multiple models & embeddingsCopy Anchor Link

Configure dedicated models for fast completions, reasoning, and vector embeddings:

ai: {
  models: [
    {
      id: 'default',
      name: 'GPT-4o Mini',
      model: openai('gpt-4o-mini'),
      capabilities: ['text', 'structured-output', 'streaming'],
    },
    {
      id: 'reasoning',
      name: 'Claude Sonnet',
      model: 'anthropic/claude-sonnet-5', // AI Gateway id
      capabilities: ['text', 'image-input', 'structured-output', 'streaming'],
    },
  ],
  embeddingModels: [
    { id: 'default', name: 'Text Embedding 3', model: openai.embedding('text-embedding-3-small') },
  ],
}

Resolve specific models by ID:

const smartModel = c.get('ai').model('reasoning')
const embedModel = c.get('ai').embeddingModel()
const capabilities = c.get('ai').capabilities('reasoning') // ['text', 'image-input', ...]

model(), embeddingModel(), imageModel() and models() work as before. Admins can assign a different model (and a fallback) to each action in the AdminCP; an assigned model that lacks a required capability is refused.

Reading the model list in the browserCopy Anchor Link

Provider instances, API keys and prices stay on the server. c.get("ai").models() returns only the public half of each entry:

The middleware route publishes this list, and the browser reads it with useMiddlewareConfigQuery():

import { useMiddlewareConfigQuery } from '@vitnode/core/tanstack/auth'

export const ModelPicker = () => {
  const { data } = useMiddlewareConfigQuery()

  if (data.ai.models.length === 0) return <p>AI is not configured.</p>

  return (
    <select className="w-full">
      {data.ai.models.map((model) => (
        <option key={model.id} value={model.id}>
          {model.name}
        </option>
      ))}
    </select>
  )
}

The first entry is the default, which c.get("ai").model() resolves when a handler gets no id. The list is empty when no ai.models are configured, so a picker can hide itself instead of offering nothing.

Learn moreCopy Anchor Link