AI Engine
Built-in AI with createGencowAI() and ai.* — chat, vision, image generation, embeddings, reranking, speech-to-text, text-to-speech, structured output, model catalog, and service-credit billing
Gencow includes a built-in AI engine powered by the Vercel AI SDK and the
Gencow Platform AI proxy. After gencow add AI, import from the generated
gencow/ai.ts file.
In local development, chat, embedding, image, and vision calls can go directly to
OpenAI with OPENAI_API_KEY. After cloud deployment, they automatically use the
Gencow proxy, so tenant apps do not manage provider API keys. Speech helpers use
the Gencow proxy because private storage, preprocessing, async jobs, and service
credit billing are platform-owned.
Quick Setup
# Add the AI component
npx gencow@latest add AIThis creates several generated files under gencow/. Import from gencow/ai.ts
— that is the public entry point for both APIs described below.
Two APIs
gencow/ai.ts exposes two ways to call AI:
| Provider API | Facade API | |
|---|---|---|
| Entry point | createGencowAI() + Vercel AI SDK (generateText, embed, rerank, ...) |
import { ai } from "./ai" |
| Best for | New application code, agents, tools, local direct-provider streaming, feature-flagged cloud text streaming, full SDK surface | Existing Gencow components and simple chat/embed/image/vision/speech handlers |
| Return types | Standard AI SDK result types | Normalized Gencow shapes with a few extra fields |
Use the provider API for new code. The facade API remains available for compatibility, but it only wraps a narrow set of common cases — chat, bounded text/image input, vision text extraction, embeddings, reranking, structured output, and single-image generation. Speech helpers also live on the facade because they depend on platform storage and async job orchestration. It does not expose the full AI SDK feature set (for example advanced tool loops or provider options). It is not deprecated today, but new features usually land on the provider path first.
Runtime Contract
Deployed apps use OpenAI-compatible platform routes:
| Capability | Platform route | Cloud support |
|---|---|---|
| Chat completions | /platform/ai/v1/chat/completions |
Text and image-input chat; text streaming when the platform enables it |
| Vision text extraction | /platform/ai/v1/chat/completions |
ai.vision.extractText() helper |
| Embeddings | /platform/ai/v1/embeddings |
Supported |
| Reranking | /platform/ai/v1/rerank |
AI SDK rerank() and ai.rerank() |
| Image generation | /platform/ai/v1/images/generations |
Single-image generation |
| Speech-to-text | /platform/ai/v1/speech/transcriptions + /platform/ai/v1/audio/jobs/:jobId |
Async job from a private storageId |
| Text-to-speech | /platform/ai/v1/speech/synthesis |
Private audio file output, optional read grant |
| Proxy health | /platform/ai/health |
Supported |
Streaming Support
Cloud text streaming is available only when the platform has enabled
GENCOW_AI_STREAM_ENABLED for the active environment. Treat it as a capability,
not as a hardcoded assumption in your app.
| Request shape | Cloud behavior |
|---|---|
| Non-streaming text chat | Supported |
| Non-streaming image-input chat | Supported |
| Text streaming | Supported when platform streaming is enabled |
| Image-input or multimodal streaming | Not supported in this phase |
Use a non-streaming fallback for deployed apps that must work across every environment:
import { generateText, streamText } from "ai";
import { createGencowAI } from "./ai";
const gencow = createGencowAI();
function isStreamingUnavailable(error: unknown): boolean {
const message = error instanceof Error ? error.message : String(error);
return /stream|unsupported|disabled/i.test(message);
}
type ChatMessage = { role: "user"; content: string };
export async function answerWithOptionalStreaming(messages: ChatMessage[], options = { preferStreaming: false }) {
if (options.preferStreaming) {
try {
const stream = streamText({
model: gencow.languageModel("gpt-5.4-mini"),
messages,
});
return { kind: "stream" as const, stream: stream.textStream };
} catch (error) {
// Fall back when the deployed platform has streaming disabled.
// Do not log prompts, tokens, provider keys, or private file URLs here.
if (!isStreamingUnavailable(error)) {
throw error;
}
}
}
const result = await generateText({
model: gencow.languageModel("gpt-5.4-mini"),
messages,
});
return { kind: "text" as const, text: result.text };
}Local direct-provider streaming continues to work when you run without a platform proxy URL and have the required local provider credentials. In cloud, prefer capability detection plus fallback instead of assuming every deployment has streaming enabled.
Provider API
Create a provider once per handler (or module) and pass its models into standard AI SDK functions:
import { embed, generateImage, generateText, rerank } from "ai";
import { createGencowAI } from "./ai";
const gencow = createGencowAI();Results use AI SDK types directly — for example GenerateTextResult from
generateText() exposes text and usage with the standard
LanguageModelUsage shape.
Chat in Mutations
import { v } from "@gencow/core";
import { generateText } from "ai";
import { procedure } from "./runtime";
import { createGencowAI } from "./ai";
export const chat = procedure.mutation
.name("chat.send")
.input(v.object({
messages: v.array(v.object({
role: v.string(),
content: v.string(),
})),
}))
.handler(async ({ context: ctx, input }) => {
ctx.auth.requireAuth();
const gencow = createGencowAI();
const result = await generateText({
model: gencow.languageModel("gpt-5.4-mini"),
system: "You are a concise support assistant.",
messages: input.messages,
});
return {
role: "assistant",
content: result.text,
usage: result.usage,
};
});Pick an explicit model in production so quality/cost tradeoffs are intentional.
Model Selection
import { generateText } from "ai";
import { createGencowAI } from "./ai";
const gencow = createGencowAI();
// Highest-quality reasoning/coding path
await generateText({
model: gencow.languageModel("gpt-5.4"),
messages: [{ role: "user", content: "Review this architecture..." }],
});
// Strong default for most production chat, coding, and agent steps
await generateText({
model: gencow.languageModel("gpt-5.4-mini"),
messages: [{ role: "user", content: "Draft the reply." }],
});
// High-volume classification/extraction
await generateText({
model: gencow.languageModel("gpt-5.4-nano"),
messages: [{ role: "user", content: "Classify this ticket." }],
});
// Compatibility with older generated examples
await generateText({
model: gencow.languageModel("gpt-5-mini"),
messages: [{ role: "user", content: "안녕?" }],
});Structured Output
Use generateObject() when the handler needs typed JSON. Do not ask a model to
return JSON text and then call JSON.parse().
import { v } from "@gencow/core";
import { generateObject } from "ai";
import { procedure } from "./runtime";
import { z } from "zod";
import { createGencowAI } from "./ai";
const analysisSchema = z.object({
sentiment: z.enum(["positive", "negative", "neutral"]),
score: z.number().min(0).max(1),
keywords: z.array(z.string()),
summary: z.string(),
});
export const analyze = procedure.mutation
.name("tasks.analyze")
.input(v.object({ text: v.string() }))
.handler(async ({ context: ctx, input }) => {
ctx.auth.requireAuth();
const gencow = createGencowAI();
const { object } = await generateObject({
model: gencow.languageModel("gpt-5.4-mini"),
system: "Analyze the text and extract structured data.",
schema: analysisSchema,
prompt: input.text,
});
return object;
});Avoid
generateText()+JSON.parse(): LLMs can include Markdown fences or violate the schema.generateObject()uses SDK-level structured output and Zod validation.
Image Generation
import { generateImage } from "ai";
import { createGencowAI } from "./ai";
const gencow = createGencowAI();
const result = await generateImage({
model: gencow.imageModel("gpt-image-2"),
prompt: "A clean product shot of a stainless steel water bottle",
});
const image = result.images[0];
// image.base64, image.mediaType — standard AI SDK GenerateImageResult shapeImage generation uses gpt-image-2 by default when you pass that model id. In
local development it calls OpenAI directly with OPENAI_API_KEY; in cloud it uses
the Gencow AI proxy and charges service credits.
Embeddings
import { embed } from "ai";
import { createGencowAI } from "./ai";
const gencow = createGencowAI();
const { embedding } = await embed({
model: gencow.embeddingModel("text-embedding-3-small"),
value: "Hello, world!",
});
// number[]; text-embedding-3-small currently returns 1536 dimensionspgvector schema match required: if you store embeddings in a pgvector column, the selected embedding model's output dimension must match the column size. The generated
rag_documents.embeddingandagent_memories.embeddingstarter schemas usevector(1536), so swapping to a different embedding dimension requires a schema change plus re-indexing/migration of existing vectors.
For bulk document search, prefer gencow add RAG and the canonical
documents.ingest.* flow. It calls /platform/ai/v1/embeddings through the
platform path and keeps indexing, metering, and visibility scope consistent.
Reranking
Use the AI SDK rerank() helper with the generated Gencow provider when you
need a dedicated reranking model in a custom search pipeline:
import { rerank } from "ai";
import { createGencowAI } from "./ai";
const gencow = createGencowAI();
const result = await rerank({
model: gencow.rerankingModel("Cohere-rerank-v4.0-fast"),
query: "refund policy",
documents: [
"Refunds are available within 30 days.",
"Invoices are emailed after payment.",
],
topN: 1,
});gencow.rerankingModel(...) requires the Gencow AI proxy route. Local direct
mode with only OPENAI_API_KEY does not support real reranking yet and
should fail fast instead of silently degrading.
gencow add Reranker wraps this provider path behind a generated
createReranker() factory and a backward-compatible reranker singleton. The
starter can keep an explicit LLM fallback path for compatibility, but the pure
provider surface itself remains proxy-only.
Facade API
The facade wraps the provider API behind ai.* helpers. Use it when maintaining
existing Gencow components or writing straightforward handlers that fit its
narrow surface area.
It is not deprecated, but it is a compatibility layer — not the long-term home for new AI features. Prefer the provider API when you need advanced tools, feature-flagged cloud text streaming, local direct-provider streaming, provider-specific options, or any AI SDK capability beyond simple chat, image input, embed, rerank, structured output, image generation, and speech helpers.
import { ai } from "./ai";Chat in Mutations
import { v } from "@gencow/core";
import { procedure } from "./runtime";
import { ai } from "./ai";
export const chat = procedure.mutation
.name("chat.send")
.input(v.object({
messages: v.array(v.object({
role: v.string(),
content: v.string(),
})),
}))
.handler(async ({ context: ctx, input }) => {
ctx.auth.requireAuth();
const result = await ai.chat({
system: "You are a concise support assistant.",
messages: input.messages,
model: "gpt-5.4-mini",
});
return {
role: "assistant",
content: result.text,
usage: result.usage,
creditsCharged: result.creditsCharged,
};
});You can omit model to use the helper default (gpt-5.4-mini). For production
workloads, pick an explicit model so quality/cost tradeoffs are intentional.
Model Selection
import { ai } from "./ai";
// Highest-quality reasoning/coding path (Startup and Enterprise)
await ai.chat({
model: "gpt-5.4",
messages: [{ role: "user", content: "Review this architecture..." }],
});
// Strong default for most production chat, coding, and agent steps
await ai.chat({
model: "gpt-5.4-mini",
messages: [{ role: "user", content: "Draft the reply." }],
});
// High-volume classification/extraction
await ai.chat({
model: "gpt-5.4-nano",
messages: [{ role: "user", content: "Classify this ticket." }],
});
// Compatibility with older generated examples
await ai.chat({
model: "gpt-5-mini",
messages: [{ role: "user", content: "안녕?" }],
});Image-to-Text / Vision
ai.chat() accepts bounded text/image content parts. Use it when the prompt
needs to reason over an image:
import { ai } from "./ai";
const result = await ai.chat({
model: "gpt-5.4-mini",
messages: [
{
role: "user",
content: [
{ type: "text", text: "Extract the visible text as Markdown." },
{
type: "image",
image: receiptBase64,
mediaType: "image/png",
detail: "high",
},
],
},
],
});
console.log(result.text);For plain OCR-like extraction, use ai.vision.extractText():
import { ai } from "./ai";
const extracted = await ai.vision.extractText({
image: receiptBase64,
mediaType: "image/png",
output: "markdown",
});
console.log(extracted.markdown ?? extracted.text);Supported input media types are image/png, image/jpeg, image/webp, and
image/gif. Cloud requests may include up to 5 images and 20 MiB total image
bytes. External HTTP image URLs and file content parts are not supported in the
first version; pass a base64 string, Uint8Array, ArrayBuffer, or data URL
instead.
Structured Output
import { v } from "@gencow/core";
import { procedure } from "./runtime";
import { z } from "zod";
import { ai } from "./ai";
export const analyze = procedure.mutation
.name("tasks.analyze")
.input(v.object({ text: v.string() }))
.handler(async ({ context: ctx, input }) => {
ctx.auth.requireAuth();
const { object } = await ai.generateObject({
model: "gpt-5.4-mini",
system: "Analyze the text and extract structured data.",
schema: z.object({
sentiment: z.enum(["positive", "negative", "neutral"]),
score: z.number().min(0).max(1),
keywords: z.array(z.string()),
summary: z.string(),
}),
prompt: input.text,
});
return object;
});Avoid
ai.chat()+JSON.parse(): Useai.generateObject()instead.
Image Generation
import { ai } from "./ai";
const icon = await ai.image.generate({
prompt: "A clean app icon for a project management product",
model: "gpt-image-2",
size: "1024x1024",
quality: "low",
format: "png",
});
return {
base64: icon.images[0].base64,
mimeType: icon.images[0].mimeType,
creditsCharged: icon.creditsCharged,
};ai.image.generate() uses gpt-image-2 by default. The helper does not fall
back from cloud proxy mode to direct OpenAI calls.
Embeddings
import { ai } from "./ai";
const embedding = await ai.embed("Hello, world!");
// number[]; text-embedding-3-small currently returns 1536 dimensionspgvector schema match required:
ai.embed()still uses an embedding model under the hood. If you persist the vector in pgvector, keep the model dimension aligned with your database column size. The generated starter RAG and Memory schemas currently usevector(1536).
For multiple texts in one call, use ai.embedMany(texts).
Reranking
Use ai.rerank() after retrieval has already enforced owner, app,
visibility, and read-grant checks. Gencow sends only query and
documents[].text to the provider. documents[].metadata is ignored by the
gateway and must not be used as an authorization source.
import { ai } from "./ai";
const ranked = await ai.rerank({
query: "refund policy for startup plan",
documents: candidates.map((candidate) => ({
id: candidate.id,
text: candidate.text,
})),
topK: 5,
providerPreference: "auto",
});
ranked.provider // "azure_cohere" or "openai"
ranked.model // "Cohere-rerank-v4.0-fast" or "gpt-5.4-mini"
ranked.fallbackUsed // true when GPT fallback was used
ranked.results // [{ index, id?, relevanceScore }]Default routing is Azure Cohere Cohere-rerank-v4.0-fast. When the Azure
Cohere deployment is not configured, or a transient 408/429/5xx provider
failure occurs in providerPreference: "auto" mode, the gateway uses
gpt-5.4-mini as a JSON-only fallback and returns
_gencow.fallbackUsed=true. Explicit providerPreference: "azure_cohere"
fails closed when the primary route or credentials are unavailable, and Azure
401/403 errors do not silently fall back to GPT.
Rerank usage is metered as ai_rerank, separate from chat, embeddings, and
image generation. Logs and usage metadata record counts, token estimates,
provider/model, route id, and fallback reason; they do not record raw document
text or provider keys.
Speech-to-Text and Text-to-Speech
Use ai.speech.* for audio. Speech calls must go through the Gencow proxy; do
not call Azure OpenAI, OpenAI, or ffmpeg directly from tenant app code.
Speech-to-text starts an async job from an existing private storage object. Upload
the audio or video through app storage first, then pass its storageId:
import { ai } from "./ai";
const job = await ai.speech.transcribe({
storageId: uploadedAudio.storageId,
model: "gpt-4o-mini-transcribe",
language: "ko",
responseFormat: "json",
idempotencyKey: requestId,
});
const transcriptJob = await ai.speech.waitForTranscript(job.id, {
timeoutMs: 10 * 60 * 1000,
pollIntervalMs: 2000,
});
return {
status: transcriptJob.status,
transcriptStorageId: transcriptJob.transcriptStorageId,
result: transcriptJob.result,
};ai.speech.transcribe() returns immediately with a job. Poll with
ai.speech.getTranscriptJob(job.id) or ai.speech.waitForTranscript(job.id).
Successful jobs expose transcriptStorageId and safe result metadata. If the
browser needs to download a private transcript artifact, create a short-lived
read grant from your backend before returning a URL to the client.
Text-to-speech returns a private audio object. Set includeReadGrant: true only
when your handler needs to return a short-lived browser download URL:
import { ai } from "./ai";
const narrationText = getNarrationTextFromYourApp();
const audio = await ai.speech.synthesize({
input: narrationText,
model: "gpt-4o-mini-tts",
voice: "alloy",
format: "mp3",
includeReadGrant: true,
});
return {
storageId: audio.storageId,
contentType: audio.contentType,
bytes: audio.bytes,
downloadUrl: audio.downloadUrl,
};Speech limits are platform-wide:
| Capability | Limit |
|---|---|
| STT source object | 500 MiB max |
| STT source duration | 2 hours max |
| STT direct provider threshold | 25 MiB; larger files are preprocessed and chunked |
| TTS input | 4,096 characters max |
| TTS instructions | 2,048 characters max |
| TTS output | 32 MiB max |
Supported STT options include responseFormat: "json" | "text" | "srt" | "verbose_json" | "vtt" and timestampGranularities: ["word"],
["segment"], or both. TTS supports format: "mp3" | "opus" | "aac" | "flac" | "wav" | "pcm" and voices from the generated AiSpeechVoice type.
Long STT inputs are handled by the platform media worker: the worker extracts audio, downsamples when needed, chunks safely, calls Azure OpenAI, and captures the final transcript back into private app storage. Tenant apps should treat this as an async workflow and keep their own UI in a queued/running/done state.
Return Type
ai.chat() returns a normalized object, not a raw string:
const result = await ai.chat({ messages: [{ role: "user", content: "Hi" }] });
result.text // AI response text
result.usage.totalTokens // Token usage
result.creditsCharged // Informational facade field; gateway billing is authoritative
result.model // Model name used
// Wrong: result is an object
console.log(result);
// Correct
console.log(result.text);API Key Setup
Local Development
Add your OpenAI key to .env:
OPENAI_API_KEY=sk-...Local calls go directly to OpenAI and do not charge Gencow service credits. Speech helpers are the exception: they require the Gencow AI proxy because they depend on private storage lookup, preprocessing, async job state, and service credit capture.
Cloud Deployment
No provider key is required in tenant app code or tenant app environment
variables. The platform injects GENCOW_AI_PROXY_URL,
GENCOW_AI_PROXY_URL_ALT, and GENCOW_AI_PROXY_TOKEN at runtime.
Do not add OPENAI_API_KEY to a deployed tenant app to bypass the proxy. That
breaks centralized key management, service-credit charging, and usage reporting.
Supported Models
The displayed credit rates are base service credits per 1K tokens before plan markup. Internally, the gateway divides these rates by 1000 and multiplies by the actual input/output token counts.
| Model | Plan | Best fit | Input cr / 1K | Output cr / 1K |
|---|---|---|---|---|
gpt-5.4 |
Pro/Scale only | Highest-quality Azure-first reasoning, coding, complex professional work | 25 | 150 |
gpt-5.4-mini |
Free/Pro/Scale | Recommended strong default for production chat, coding, agents | 7.5 | 45 |
gpt-5.4-nano |
Free/Pro/Scale | Simple high-volume extraction, ranking, classification | 2 | 12.5 |
gpt-5-mini |
Free/Pro/Scale | Compatibility replacement for deprecated mini-class requests such as gpt-4o-mini |
2.5 | 20 |
gpt-5-nano |
Free/Pro/Scale | Compatibility low-cost path for deprecated nano-class requests | 0.5 | 4 |
Cohere-rerank-v4.0-fast |
Free/Pro/Scale | Default dedicated reranker for search/RAG candidate ordering | 1.5 | 0 |
Cohere-rerank-v4.0-pro |
Free/Pro/Scale | Higher-quality dedicated reranker route when enabled | 4.5 | 0 |
Embedding models:
| Model | Best fit | Input cr / 1K | Output cr / 1K |
|---|---|---|---|
text-embedding-3-small |
Default RAG/search embedding | 0.2 | 0 |
text-embedding-3-large |
Higher-quality embedding when cost is acceptable | 1.3 | 0 |
When storing embeddings in pgvector, choose a model whose output dimension
matches your column definition. The generated starter schemas use
vector(1536).
Image models:
| Model | Best fit | Notes |
|---|---|---|
gpt-image-2 |
Default image generation path | Supports quality: "low" for cheaper smoke/testing |
gpt-image-1.5 |
Compatibility/fallback image path | Uses legacy image size options |
gpt-image-1-mini |
Lower-cost image generation | Useful for smoke tests and budget-sensitive apps |
Audio models:
| Model | Operation | Plan | Meter | Base cr / unit |
|---|---|---|---|---|
gpt-4o-mini-transcribe |
Speech-to-text | Free/Pro/Scale | audio second | 0.5 |
gpt-4o-transcribe |
Speech-to-text | Free/Pro/Scale | audio second | 1 |
gpt-4o-mini-tts |
Text-to-speech | Free/Pro/Scale | input character | 0.006 |
Audio price meters are seeded from Azure OpenAI pricing. Platform billing is authoritative: the app should read returned usage/job metadata for display, but must not calculate final charges client-side.
Image generation supports single-image n=1 requests in MVP. moderation is
server-gated to auto; low moderation is not exposed until platform policy and
plan gating exist. The proxy validates provider b64_json output before
returning it and rejects oversized images with a typed error.
Choosing a Model
| Requirement | Recommended model |
|---|---|
| Best possible answer quality | gpt-5.4 on Pro/Scale |
| Strong production default | gpt-5.4-mini |
| Cheapest high-volume GPT-5-class path | gpt-5.4-nano |
| Existing 4o-era app compatibility | Request gpt-4o-mini, which is automatically run as gpt-5-mini |
| Vector search/RAG embeddings | text-embedding-3-small |
| Image generation | gpt-image-2; use gpt-image-1-mini for lower-cost smoke tests |
| Speech-to-text default | gpt-4o-mini-transcribe |
| Higher-quality speech-to-text | gpt-4o-transcribe |
| Text-to-speech | gpt-4o-mini-tts |
OpenAI recommends starting with the newest frontier model for complex reasoning and smaller variants for latency/cost-sensitive work. Gencow follows that shape but lets platform admins disable or reprice a model without tenant code changes.
Service-Credit Billing
Gencow uses service credits for AI and other provider-backed services.
Base calculation:
baseCredits =
inputTokens * inputCrPerToken +
outputTokens * outputCrPerToken
creditsCharged = baseCredits * plan.serviceMarkupDefault plan markups:
| Plan | AI service markup |
|---|---|
| Hobby | 1.5x |
| Startup | 0.8x |
| Enterprise | Custom |
Example with gpt-5.4-mini, Startup plan, 1,000 input tokens and 1,000 output
tokens:
base = 7.5 + 45 = 52.5 credits
charged = 52.5 * 0.8 = 42 creditsIf service credits are exhausted or a spend cap blocks the request, the proxy returns a 402 response before or after provider execution depending on whether a reservation was possible.
Image generation records ai_image_input_tokens, ai_image_output_tokens, and
ai_image_count service-usage metrics so image costs do not mix into generic AI
chat token rows.
Image-input chat and ai.vision.extractText() are metered through the chat
service-credit path. The gateway records safe request shape only, such as text
character count, image count, image bytes, media types, and detail. It must not
store raw base64 images, prompts, read-grant URLs, bearer tokens, or provider
secrets in usage events or logs.
Speech-to-text reserves/captures service credits from the audio_seconds meter
after the platform has probed or preprocessed the source. Text-to-speech reserves
from the input_characters meter before provider execution and captures the
final usage event after storing the private audio output. If a provider call
fails after reservation, the platform refunds or marks the job with the matching
terminal status.
Error Handling
Common cloud proxy responses:
| Status | Cause | Fix |
|---|---|---|
| 400 | Unsupported model |
Use an active model from model_pricing |
| 400 | Streaming requested while platform streaming is disabled or unsupported for this request shape | Use generateText() / ai.chat() without streaming, or retry text streaming only after GENCOW_AI_STREAM_ENABLED is active |
| 400 | Unsupported image media type, external image URL, or file content part | Use base64/data URL image/png, image/jpeg, image/webp, or image/gif input |
| 400 | Missing or invalid speech storageId, language, timestamp granularity, voice, or audio format |
Upload to private app storage first and pass only supported speech options |
| 401/403 | Missing or invalid proxy/app token | Redeploy or reinstall AI component |
| 402 | Service credits exhausted, spend cap exceeded, or gpt-5.4 effective model requested from Free/compatibility-substituted gpt-5.5 |
Charge credits, disable cap, use a cheaper model, or upgrade plan |
| 413 | Image input exceeds the 20 MiB request cap | Compress or resize the image before sending it |
| 413 | Generated image exceeds platform byte cap | Retry with smaller/lower-quality output |
| 413 | Speech source or TTS input/output exceeds platform caps | Use a smaller source, shorter input, or split the workflow |
| 429/503 | STT preprocessing capacity or provider route is temporarily unavailable | Keep the job/request retryable and retry with backoff |
| 500/502 | Provider or platform transient failure | Retry with backoff; check platform health/logs |
Do Not Call LLM APIs Directly
// Wrong: provider SDK/key management bypasses Gencow
import OpenAI from "openai";
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
await openai.chat.completions.create({
model: "gpt-5.4-mini",
messages: [{ role: "user", content: "Hello" }],
});
// Correct: generated Gencow AI helper
import { generateText } from "ai";
import { createGencowAI } from "./ai";
const gencow = createGencowAI();
const result = await generateText({
model: gencow.languageModel("gpt-5.4-mini"),
messages: [{ role: "user", content: "Hello" }],
});
console.log(result.text);The same rule applies to speech: upload media to Gencow storage, call
ai.speech.transcribe() or ai.speech.synthesize(), and let the platform handle
Azure OpenAI routing, media preprocessing, private artifact storage, and service
credits.
Why Not Direct LLM Calls?
| Direct SDK | createGencowAI() / import { ai } |
|
|---|---|---|
| API key | Each app manages provider keys | Platform manages provider keys |
| Cloud billing | Not tracked by Gencow | Service-credit charging and usage snapshots |
| Spend caps | App must implement | Platform enforced |
| Supported model list | Hardcoded in app | Active model_pricing rows |
| RAG/Memory | App must assemble manually | gencow add RAG / gencow add Memory |
| Guardrails | App must assemble manually | gencow add Guardrails |
Quick Decision Tree
Need AI?
|
|-- Chat / text response -> gencow add AI -> createGencowAI() + generateText()
|-- Typed JSON extraction -> gencow add AI -> createGencowAI() + generateObject()
|-- Image generation -> gencow add AI -> createGencowAI() + generateImage()
|-- Reranking candidates -> gencow add AI -> createGencowAI() + rerank()
|-- Speech-to-text -> gencow add AI -> ai.speech.transcribe() + waitForTranscript()
|-- Text-to-speech -> gencow add AI -> ai.speech.synthesize()
|-- Document search / RAG -> gencow add RAG -> rag.retrieve() / ctx.search()
|-- Agent memory -> gencow add Memory -> memory.buildContext()
|-- Safety filtering -> gencow add Guardrails -> guardrails.validateInput()
`-- Vector embedding -> gencow add AI -> createGencowAI() + embed()# Do not install provider SDKs directly for app AI calls
npm install openai
npm install @anthropic-ai/sdk
npm install @google/generative-ai
# Use Gencow components
npx gencow@latest add AI
npx gencow@latest add RAG
npx gencow@latest add MemoryNext Steps
- Components — All
gencow addcomponents - RAG & Memory — Document search and agent memory
- Local Development — Local
.envsetup