AI Engine

Built-in AI with createGencowAI() and ai.* — OpenAI chat, embeddings and speech, Azure Cohere reranking, model selection, and service-credit billing

Gencow includes a built-in AI engine powered by the Vercel AI SDK and the Gencow Platform AI proxy. After gencow add AI, import from the generated gencow/ai.ts file.

In local development, chat, embedding, image, and vision calls can go directly to OpenAI with OPENAI_API_KEY. After cloud deployment, they automatically use the Gencow proxy, so tenant apps do not manage provider API keys. Speech helpers use the Gencow proxy because private storage, preprocessing, async jobs, and service credit billing are platform-owned.

Quick Setup

# Add the AI component
bunx gencow@latest add AI

This creates several generated files under gencow/. Import from gencow/ai.ts — that is the public entry point for both APIs described below.

Two APIs

gencow/ai.ts exposes two ways to call AI:

Use the provider API for new code. The facade API remains available for compatibility, but it only wraps a narrow set of common cases — chat, bounded text/image input, vision text extraction, embeddings, reranking, structured output, and single-image generation. Speech helpers also live on the facade because they depend on platform storage and async job orchestration. It does not expose the full AI SDK feature set (for example advanced tool loops or provider options). It is not deprecated today, but new features usually land on the provider path first.

Runtime Contract

Deployed apps use the platform-managed AI proxy. Call it only through createGencowAI() or the generated ai facade; app code must not depend on platform-internal route paths, headers, provider endpoints, or keys.

Streaming Support

Cloud text streaming is available only when the active environment exposes that capability. Detect it at runtime instead of treating it as a hardcoded assumption.

Use a non-streaming fallback for deployed apps that must work across every environment:

import { generateText, streamText } from "ai";
import { createGencowAI } from "./ai";

const gencow = createGencowAI();

function isStreamingUnavailable(error: unknown): boolean {
    const message = error instanceof Error ? error.message : String(error);
    return /stream|unsupported|disabled/i.test(message);
}

type ChatMessage = { role: "user"; content: string };

export async function answerWithOptionalStreaming(messages: ChatMessage[], options = { preferStreaming: false }) {
    if (options.preferStreaming) {
        try {
            const stream = streamText({
                model: gencow.languageModel("llm/balanced"),
                messages,
            });

            return { kind: "stream" as const, stream: stream.textStream };
        } catch (error) {
            // Fall back when the deployed platform has streaming disabled.
            // Do not log prompts, tokens, provider keys, or private file URLs here.
            if (!isStreamingUnavailable(error)) {
                throw error;
            }
        }
    }

    const result = await generateText({
        model: gencow.languageModel("llm/balanced"),
        messages,
    });

    return { kind: "text" as const, text: result.text };
}

Local direct-provider streaming continues to work when you run without a platform proxy URL and have the required local provider credentials. In cloud, prefer capability detection plus fallback instead of assuming every deployment has streaming enabled.

Use Virtual Chat Profiles by Default

For a simple default, choose a stable llm/* alias. You can also pass an OpenAI model ID directly to hosted calls; Gencow forwards that ID without a model allowlist or silent substitution. OpenAI availability and deprecation behavior therefore apply. A model/capability without an active price snapshot is rejected before provider execution.

Aliases are convenience defaults; direct OpenAI IDs remain available when an application needs a particular OpenAI model or capability. The aliases currently resolve as follows: llm/economy → gpt-6-luna, llm/balanced → gpt-6-sol, llm/quality → gpt-6-astra, and llm/code → gpt-6-luna. OpenAI model requests are not silently moved to another model when deprecated or unavailable.

Hosted chat requests that omit model use llm/economy. Earlier direct model IDs such as gpt-5.6-sol, gpt-5.6-terra, gpt-5.6-luna, and gpt-5.4-mini have not been removed by the alias change. They remain direct OpenAI requests while OpenAI serves them and Gencow has an active price for the requested capability. A listed model or a price row alone does not guarantee provider access for every account or request.

Virtual profiles are resolved by the hosted Gencow proxy. When running locally with only OPENAI_API_KEY, use a concrete OpenAI model ID because OpenAI does not recognize Gencow's llm/* names. Never copy that local-only model pin into new hosted application code.

Provider API

Create a provider once per handler (or module) and pass its models into standard AI SDK functions:

import { embed, generateImage, generateText, rerank } from "ai";
import { createGencowAI } from "./ai";

const gencow = createGencowAI();

Results use AI SDK types directly — for example GenerateTextResult from generateText() exposes text and usage with the standard LanguageModelUsage shape.

Chat in Mutations

import { v } from "@gencow/core";
import { generateText } from "ai";
import { procedure } from "./runtime";
import { createGencowAI } from "./ai";

export const chat = procedure.mutation
    .name("chat.send")
    .input(v.object({
        messages: v.array(v.object({
            role: v.string(),
            content: v.string(),
        })),
    }))
    .handler(async ({ context: ctx, input }) => {
        ctx.auth.requireAuth();

        const gencow = createGencowAI();
        const result = await generateText({
            model: gencow.languageModel("llm/balanced"),
            system: "You are a concise support assistant.",
            messages: input.messages,
        });

        return {
            role: "assistant",
            content: result.text,
            usage: result.usage,
        };
    });

Pick an explicit virtual profile in hosted production code so the intended quality/cost position is clear without pinning the provider model lifecycle.

Model Selection

import { generateText } from "ai";
import { createGencowAI } from "./ai";

const gencow = createGencowAI();

// Highest-quality reasoning path
await generateText({
    model: gencow.languageModel("llm/quality"),
    messages: [{ role: "user", content: "Review this architecture..." }],
});

// Stable default profile for general Startup/Enterprise workloads
await generateText({
    model: gencow.languageModel("llm/balanced"),
    messages: [{ role: "user", content: "Draft the reply." }],
});

// High-volume classification/extraction
await generateText({
    model: gencow.languageModel("llm/economy"),
    messages: [{ role: "user", content: "Classify this ticket." }],
});

// Coding and authorized function tools
await generateText({
    model: gencow.languageModel("llm/code"),
    messages: [{ role: "user", content: "Review this patch." }],
});

Structured Output

Use generateObject() when the handler needs typed JSON. Do not ask a model to return JSON text and then call JSON.parse().

import { v } from "@gencow/core";
import { generateObject } from "ai";
import { procedure } from "./runtime";
import { z } from "zod";
import { createGencowAI } from "./ai";

const analysisSchema = z.object({
    sentiment: z.enum(["positive", "negative", "neutral"]),
    score: z.number().min(0).max(1),
    keywords: z.array(z.string()),
    summary: z.string(),
});

export const analyze = procedure.mutation
    .name("tasks.analyze")
    .input(v.object({ text: v.string() }))
    .handler(async ({ context: ctx, input }) => {
        ctx.auth.requireAuth();

        const gencow = createGencowAI();
        const { object } = await generateObject({
            model: gencow.languageModel("llm/balanced"),
            system: "Analyze the text and extract structured data.",
            schema: analysisSchema,
            prompt: input.text,
        });

        return object;
    });

Avoid generateText() + JSON.parse(): LLMs can include Markdown fences or violate the schema. generateObject() uses SDK-level structured output and Zod validation.

Image Generation

import { generateImage } from "ai";
import { createGencowAI } from "./ai";

const gencow = createGencowAI();
const result = await generateImage({
    model: gencow.imageModel("gpt-image-2"),
    prompt: "A clean product shot of a stainless steel water bottle",
});

const image = result.images[0];
// image.base64, image.mediaType — standard AI SDK GenerateImageResult shape

Image generation uses gpt-image-2 by default when you pass that model id. In local development it calls OpenAI directly with OPENAI_API_KEY; in cloud it uses the Gencow AI proxy and charges service credits.

Hosted image generation uses OpenAI models through the Gencow proxy. Other provider model identifiers are not active runtime routes.

Embeddings

import { embed } from "ai";
import { createGencowAI } from "./ai";

const gencow = createGencowAI();
const { embedding } = await embed({
    model: gencow.embeddingModel("text-embedding-3-small"),
    value: "Hello, world!",
});
// number[]; text-embedding-3-small currently returns 1536 dimensions

[!IMPORTANT] Model and vector size must match. Embedding models return vectors with a fixed length, and pgvector columns are fixed-width. If the model output length and column definition differ, inserts and similarity queries fail. The current default starter profile uses text-embedding-3-small, which matches the default 1536 dimensions. The generated starters handle this with explicit embedding profiles such as RAG_EMBEDDING_PROFILE, MEMORY_EMBEDDING_PROFILE, and SEARCH_EMBEDDING_PROFILE. These profiles define the model id and dimensions used by schema helpers and runtime validation. If you change the embedding model, you must change the dimensions to match it, then update the schema and rebuild existing vectors.

For bulk document search, prefer gencow add RAG and the canonical documents.ingest.* flow. It uses platform-managed embeddings and keeps indexing, metering, and visibility scope consistent.

Reranking

Use the hosted Azure Cohere reranker after retrieval has already applied your app's access checks. The default model is Cohere-rerank-v4.0-fast; the Pro price is listed below for reference, but the Pro provider route is not active. The gateway does not silently replace reranking with an OpenAI chat model.

import { ai } from "./ai";

const ranked = await ai.rerank({
    query: "refund policy",
    documents: [
        { id: "refunds", text: "Refunds are available within 30 days." },
        { id: "invoices", text: "Invoices are emailed after payment." },
    ],
    topK: 1,
    providerPreference: "auto",
});

ranked.results; // [{ index, id?, relevanceScore }]

providerPreference: "auto" and "azure_cohere" use the dedicated Cohere route. "gpt_fallback" is unsupported. The provider API also supports rerank({ model: gencow.rerankingModel("Cohere-rerank-v4.0-fast"), ... }) through the hosted proxy. Local OpenAI-only mode has no rerank provider. Requests accept at most 100 documents. Only query and document text are sent for ranking; document metadata must not determine authorization.

Speech-to-Text and Text-to-Speech

Use ai.speech.* for audio. Speech calls must go through the Gencow proxy; do not call an external provider directly from tenant app code.

Speech requests normally use OpenAI routes. azure-speech-fast-transcription is an explicit Azure Speech STT option for word/phrase timestamps and diarization. Other Azure AI routes, including Azure Speech TTS, remain unavailable. See Azure Speech for the supported STT exception.

Speech-to-text starts an async job from an existing private storage object. Upload the audio or video through app storage first, then pass its storageId:

import { ai } from "./ai";

const job = await ai.speech.transcribe({
    storageId: uploadedAudio.storageId,
    model: "gpt-4o-mini-transcribe",
    language: "ko",
    responseFormat: "json",
    idempotencyKey: requestId,
});

const transcriptJob = await ai.speech.waitForTranscript(job.id, {
    timeoutMs: 10 * 60 * 1000,
    pollIntervalMs: 2000,
});

return {
    status: transcriptJob.status,
    transcriptStorageId: transcriptJob.transcriptStorageId,
    result: transcriptJob.result,
};

ai.speech.transcribe() returns immediately with a job. Poll with ai.speech.getTranscriptJob(job.id) or ai.speech.waitForTranscript(job.id). Successful jobs expose transcriptStorageId and safe result metadata. If the browser needs to download a private transcript artifact, create a short-lived read grant from your backend before returning a URL to the client.

For text-to-speech, use gpt-4o-mini-tts through the Gencow proxy. Direct synthesis stores the private audio file and can return a short-lived read grant:

import { ai } from "./ai";

const audio = await ai.speech.synthesize({
    input: "Welcome to Gencow.",
    model: "gpt-4o-mini-tts",
    voice: "alloy",
    format: "mp3",
    includeReadGrant: true,
});

audio.storageId;      // Private generated audio
audio.creditsCharged; // Final service-credit charge

For longer work, use ai.speech.createSynthesisJob() with an input stored in private app storage, then poll the returned job ID. The OpenAI speech stream now provides request-level text-input and audio-output token usage for final credit settlement. Azure Speech Neural TTS is not an active hosted route.

Return Type

ai.chat() returns a normalized object, not a raw string:

const result = await ai.chat({ messages: [{ role: "user", content: "Hi" }] });

result.text                        // AI response text
result.usage.totalTokens           // Token usage
result.creditsCharged              // Informational facade field; gateway billing is authoritative
result.model                       // Model name used

// Wrong: result is an object
console.log(result);

// Correct
console.log(result.text);

API Key Setup

Local Development

Add your OpenAI key to the backend env file:

# gencow/.env
OPENAI_API_KEY=replace-me

Local calls go directly to OpenAI and do not charge Gencow service credits. Because OpenAI does not know Gencow's llm/* IDs, local direct calls must use a concrete OpenAI model ID. Virtual profiles are resolved by the hosted Gencow AI proxy; no client-side copy of the profile map is maintained. Speech helpers are the exception: they require the Gencow AI proxy because they depend on private storage lookup, preprocessing, async job state, and service credit capture.

Cloud Deployment

No provider key is required in tenant app code or tenant app environment variables. Hosted runtimes receive platform-managed proxy access over trusted platform transport.

Do not add OPENAI_API_KEY to a deployed tenant app to bypass the proxy. That breaks centralized key management, service-credit charging, and usage reporting.

Supported Models and Price Snapshots

Hosted chat accepts OpenAI model IDs directly, subject to the capability and active price snapshot being available. It does not use a fixed model allowlist or silently substitute deprecated models. The llm/* aliases are convenience defaults:

The table shows base service credits for standard text requests, before the plan markup. Gencow uses 10,000 base credits per USD. Rates are loaded from the active, versioned OpenAI price snapshot and pinned to each request; the live snapshot, not this illustrative table, determines the charge. The figures below reflect OpenAI's standard short-context prices checked on 2026-09-25. See OpenAI API pricing for current USD rates.

Earlier OpenAI model IDs remain available as direct hosted requests when the provider accepts them and an active price snapshot exists. The following standard text base rates were present in the active Gencow price snapshot on 2026-09-29; this is a sample, not a fixed model allowlist:

The gpt-5.6-* figures use short-context rates. Cached input, cache writes, long context, and other request dimensions can change the charge. OpenAI's model lifecycle applies to direct IDs, so their continued listing does not promise permanent availability.

For requests above 272,000 input tokens, OpenAI's long-context rates apply to the full request. The price importer stores these dimensions separately. Other models and capabilities use their own imported prices and fail closed when a required price dimension is missing. OpenAI may change prices; Gencow's price sync workflow updates future snapshots, while existing requests settle against the snapshot they reserved.

RAG embeddings, image generation, speech, and reranking use separate capability prices. Azure Speech Fast Transcription and Azure Cohere Fast Rerank are the explicit non-OpenAI exceptions. When storing embeddings in pgvector, match the chosen model's output dimension to the database column.

Azure Cohere reranking is billed by search units, not tokens. One search unit generally covers one query against up to 100 documents; long inputs may consume more. Gencow settles from the provider's returned billed_units.search_units. These base rates are before plan markup:

See Microsoft's Rerank pricing.

Image generation supports single-image n=1 requests in MVP. OpenAI models accept only the server-gated moderation: "auto". The proxy validates provider b64_json output before returning it and rejects oversized images with a typed error.

Choosing a Model

Use an alias when you want a simple default; use a direct OpenAI model ID when you want to select a specific model. Direct IDs follow OpenAI's availability, deprecation, and pricing behavior.

Service-Credit Billing

Gencow uses service credits for AI and other provider-backed services.

Base calculation:

baseCredits =
  inputTokens       * inputCrPerToken +
  cachedInputTokens * cachedInputCrPerToken +
  cacheWriteTokens  * cacheWriteCrPerToken +
  outputTokens      * outputCrPerToken

creditsCharged = baseCredits * plan.serviceMarkup

Default plan markups:

Example with llm/balanced, 1,000 uncached input tokens, and 1,000 output tokens, before any account-specific billing adjustment:

base = 10 + 50 = 60 credits
Hobby: 60 * 1.5 = 90 credits
Startup or Enterprise: 60 * 1 = 60 credits

For one billed Cohere search unit, Fast costs 30 credits on Hobby and 20 on Startup/Enterprise. Pro has a 25-credit base price, but it cannot be requested until a Pro provider route is enabled. The final charge follows the actual search-unit count in the provider response. Under contention, tier admission may queue lower-priority requests; being queued does not itself consume service credits.

If service credits are exhausted or a spend cap blocks the request, the proxy returns a 402 response before or after provider execution depending on whether a reservation was possible.

Image generation records ai_image_input_tokens, ai_image_output_tokens, and ai_image_count service-usage metrics so image costs do not mix into generic AI chat token rows.

Image-input chat and ai.vision.extractText() are metered through the chat service-credit path. The gateway records safe request shape only, such as text character count, image count, image bytes, media types, and detail. It must not store raw base64 images, prompts, read-grant URLs, bearer tokens, or provider secrets in usage events or logs.

A speech-to-text reservation is an estimate, not a final charge. Exact billing requires the provider’s verified billable usage dimensions and the prices and adjustments fixed for that request. Official estimated costs per minute do not establish a per-minute billing rule for token-billed models.

OpenAI gpt-4o-mini-tts uses an input-character estimate for reservation and settles from returned text-input and audio-output token usage. Its base rates are 0.006 credit per text-input token and 0.12 credit per audio-output token, before the plan markup. See the official OpenAI TTS price. Unsupported audio models or missing required price dimensions fail before provider execution. A listed model or price row alone does not establish that every speech configuration is available.

A timeout, interrupted response, or result-storage failure does not establish that the provider did no billable work. Keep the original request or job ID, inspect its status, and follow the reported retry condition. Do not submit a new job merely because the original result or final billing status is pending; automatic recovery is not yet a verified guarantee.

Error Handling

Common cloud proxy responses:

Do Not Call LLM APIs Directly

// Wrong: provider SDK/key management bypasses Gencow
import OpenAI from "openai";
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
await openai.chat.completions.create({
    model: "gpt-6-luna",
    messages: [{ role: "user", content: "Hello" }],
});

// Correct: generated Gencow AI helper
import { generateText } from "ai";
import { createGencowAI } from "./ai";

const gencow = createGencowAI();
const result = await generateText({
    model: gencow.languageModel("llm/balanced"),
    messages: [{ role: "user", content: "Hello" }],
});
console.log(result.text);

For supported speech-to-text, upload media to Gencow storage and call ai.speech.transcribe(); the platform handles provider routing, preprocessing, private artifact storage, and service credits. For speech synthesis, use ai.speech.synthesize() or an async synthesis job with gpt-4o-mini-tts.

Why Not Direct LLM Calls?

Quick Decision Tree

Need AI?
    |
    |-- Chat / text response       -> gencow add AI -> createGencowAI() + generateText()
    |-- Typed JSON extraction      -> gencow add AI -> createGencowAI() + generateObject()
    |-- Image generation           -> gencow add AI -> createGencowAI() + generateImage()
    |-- Reranking candidates       -> gencow add AI -> ai.rerank() with Azure Cohere Fast
    |-- Speech-to-text             -> gencow add AI -> ai.speech.transcribe() + waitForTranscript()
    |-- Text-to-speech             -> gencow add AI -> ai.speech.synthesize()
    |-- Document search / RAG      -> gencow add RAG -> rag.retrieve() / ctx.search()
    |-- Durable app memory         -> gencow add Memory -> memory.remember() / memory.search()
    |-- Safety filtering           -> gencow add Guardrails -> guardrails.validateInput()
    `-- Vector embedding           -> gencow add AI -> createGencowAI() + embed()

# Do not install provider SDKs directly for app AI calls
bun add openai
bun add @anthropic-ai/sdk
bun add @google/generative-ai

# Use Gencow components
bunx gencow@latest add AI
bunx gencow@latest add RAG
bunx gencow@latest add Memory

Next Steps

• Components — All gencow add components
• Azure Speech — Fast Transcription timestamp option
• RAG & Memory — Document search and agent memory
• Local Development — Local frontend/backend env setup