Streaming

A streamed AI action shows its text while the model is still writing, so a long answer feels quick and the reader can stop it early. This page streams the Summarize topic action from AI actions into an AdminCP component, without writing a new API route.

Only output: "text" actions stream. An object action fails with AI_INVALID_INPUT when streamed.

Before you beginCopy Anchor Link

• The Summarize topic action registered in the forum plugin, from AI actions. You don't need the route you built there.
• A model with the streaming capability. A streamed run adds streaming to the action's requiredCapabilities, so only models that declare it can answer. See Model capabilities.

Stream an action into a componentCopy Anchor Link

ai: {
  models: [
    {
      id: 'default',
      name: 'GPT-4o Mini',
      model: openai('gpt-4o-mini'),
      capabilities: ['text', 'structured-output', 'streaming'], 
    },
  ],
},

import { Button } from '@vitnode/core/components/ui/button'
import { aiErrorCodeOf } from '@vitnode/core/lib/ai/assist-client'
import {
  aiAssistScope,
  streamAiAction,
} from '@vitnode/core/lib/ai/stream-client'
import React from 'react'

export const SummarizeTopic = ({
  content,
  title,
}: {
  content: string
  title: string
}) => {
  const [summary, setSummary] = React.useState('')
  const [error, setError] = React.useState<null | string>(null)
  const [pending, setPending] = React.useState(false)
  const controller = React.useRef<AbortController | null>(null)

  const onSummarize = async () => {
    controller.current = new AbortController()
    setSummary('')
    setError(null)
    setPending(true)
    try {
      await streamAiAction({
        action: '@vitnode/forum:topic.summarize',
        input: { content, title },
        onDelta: (delta) => setSummary((text) => text + delta),
        scope: aiAssistScope(), // "admin" on /admin pages, otherwise "user"
        signal: controller.current.signal,
      })
    } catch (caught) {
      setError(aiErrorCodeOf(caught)) // e.g. "AI_USER_LIMIT_REACHED"
    } finally {
      setPending(false)
    }
  }

  return (
    <div className="flex flex-col gap-2">
      <div className="flex gap-2">
        <Button isLoading={pending} onClick={onSummarize}>
          Summarize
        </Button>
        <Button
          disabled={!pending}
          onClick={() => controller.current?.abort()}
          variant="outline"
        >
          Stop
        </Button>
      </div>
      <p aria-live="polite" className="leading-relaxed">
        {summary}
      </p>
      {error && <p role="alert">{error}</p>}
    </div>
  )
}

Check the resultCopy Anchor Link

Open the AdminCP page with the component and click Summarize. The summary appears piece by piece under the buttons. Debug Panel → AI logs shows a @vitnode/forum:topic.summarize run with status succeeded.

Click Summarize again and then Stop while text is arriving. The text stops, and the AI logs show the run as canceled.

Stream from your own routeCopy Anchor Link

When the stream needs server-side work first, such as loading the topic from the database, call c.get("ai").stream() in your own route. It takes the same options as run():

handler: async (c) => {
  const { textStream } = await c.get('ai').stream({
    action: SUMMARIZE_TOPIC_AI_ACTION,
    input: c.req.valid('json'),
    signal: c.req.raw.signal, // stop paying when the reader leaves
  })

  return new Response(textStream.pipeThrough(new TextEncoderStream()), {
    headers: { 'Content-Type': 'text/plain; charset=utf-8' },
  })
}

stream() resolves to { textStream, result, runId }:

• textStream is a ReadableStream<string> of text deltas.
• result resolves to { output, runId, usage } once the stream ends, the same shape run() returns, and rejects with an AiError if the full text fails parseText() or outputSchema.

Core checks permissions, limits and budgets before stream() resolves, so a refusal is an ordinary JSON error response. To answer in the same NDJSON format as Core's routes, return aiNdjsonStream(c, request) with AI_NDJSON_HEADERS, both from @vitnode/core/api/lib/ai/stream-response.

Built-in stream routesCopy Anchor Link

streamAiAction() and estimateAiAction() call these Core routes. Their body is { action, input, idempotencyKey?, resource?, sourceFingerprint? }, the same as the run routes.

The stream routes answer application/x-ndjson, one JSON object per line:

{"t":"A tighter "}
{"t":"version of the paragraph."}
{"done":{"runId":812,"chargedPoints":"0.42","costKnown":true,"modelId":"default"}}

A failure after the stream started arrives as a last line {"e":{"code":"AI_TIMEOUT","message":"…"}}.

/estimate answers the most the run could cost, or null when no pricing bounds it. In the browser, estimateAiAction({ action, input, scope }) resolves to maxPoints as a string, or null.

What a stopped stream costsCopy Anchor Link

A stream makes one attempt on the primary model, with no retries and no fallback model. When the reader stops it, the run settles as canceled:

• If no text had reached the reader yet, they pay nothing.
• If some text had arrived, Core charges their points and daily count as for a delivered run.

RelatedCopy Anchor Link

• Quick Ask is the rich text editor feature built on these routes.
• AI errors and limits lists every error code a stream can end with.