> ## Documentation Index > Fetch the complete documentation index at: https://docs.getbifrost.ai/llms.txt > Use this file to discover all available pages before exploring further. # Evaluate state (Typesafe format) > Evaluates state against a map of named questions using Typesafe's native System One API. The request body matches Typesafe's native format; question identifiers become result field names. Images are listed in `assets` and placed by reference; send them base64 in JSON, or as file parts of a multipart request whose `request` part holds the JSON. Bifrost adds the `TypeSafe-Danger-Zone-Images` header Typesafe requires for images and box questions. ## OpenAPI ````yaml /openapi/openapi.json post /typesafe/v1/systemone openapi: 3.1.0 info: title: Bifrost API description: > Bifrost HTTP Transport API for AI model inference and gateway management. This API provides a unified interface for interacting with multiple AI providers including OpenAI, Anthropic, Bedrock, Gemini, and more through a single API, along with comprehensive management APIs for configuring and monitoring the gateway. ## API Structure ### Unified Inference API (`/v1/*`) The primary API using Bifrost's unified format. Model parameters use the format `provider/model` (e.g., `openai/gpt-4`, `anthropic/claude-3-opus`). ### Async Inference API (`/v1/async/*`) Submit inference requests for asynchronous execution. Returns a job ID immediately and allows polling for results. Supports all inference types except batches, files, and containers. ### Provider Integration APIs Native provider-format APIs for drop-in compatibility: - `/openai/*` - OpenAI-compatible API - `/anthropic/*` - Anthropic-compatible API - `/genai/*` - Google GenAI (Gemini) compatible API - `/bedrock/*` - AWS Bedrock compatible API - `/cohere/*` - Cohere compatible API ### Framework Integration APIs Multi-provider proxy endpoints for AI frameworks: - `/litellm/*` - LiteLLM proxy with all provider formats - `/langchain/*` - LangChain compatible endpoints - `/pydanticai/*` - PydanticAI compatible endpoints ### Management APIs (`/api/*`) APIs for managing and monitoring the Bifrost gateway: - `/api/config` - Configuration management - `/api/providers` - Provider and API key management - `/api/plugins` - Plugin management - `/api/governance/*` - Virtual keys, teams, customers, budgets, rate limits, routing rules, and pricing overrides - `/api/logs` - Log search and analytics - `/api/mcp/*` - MCP (Model Context Protocol) client management - `/api/session/*` - Authentication and session management - `/api/cache/*` - Cache management - `/health` - Health check endpoint ## Fallbacks Requests can include fallback models that will be tried if the primary model fails. version: 1.0.0 contact: name: Contact Us url: https://getmaxim.ai/bifrost license: name: Apache 2.0 url: https://opensource.org/licenses/Apache-2.0 servers: - url: '{baseUrl}' description: Your Bifrost instance variables: baseUrl: default: http://localhost:8080 description: Base URL of your Bifrost instance (e.g. https://bifrost.mycompany.com) security: - BearerAuth: [] - BasicAuth: [] - ApiKeyAuth: [] tags: - name: Agent Gateway description: >- Agent registrations, A2A protocol calls, push callbacks, and operation history - name: Models description: Model listing and information - name: Chat Completions description: Chat-based text generation - name: Text Completions description: Text completion generation - name: Responses description: OpenAI Responses API compatible endpoints - name: OCR description: Optical character recognition for documents and images - name: Rerank description: Document reranking by relevance to a query - name: Decisions description: Structured decisions evaluated against annotated function-tool definitions - name: Embeddings description: Text embedding generation - name: Images description: Image generations, editing, and variations - name: Videos description: Video generation and management - name: Audio description: Speech synthesis and transcription - name: Count Tokens description: Token counting utilities - name: Batch description: Batch processing operations - name: Files description: File management operations - name: Containers description: Container management operations - name: Async Jobs description: Asynchronous job submission and retrieval endpoints - name: Realtime description: Realtime WebSocket and WebRTC endpoints - name: OpenAI Integration description: OpenAI-compatible API endpoints (/openai/*) - name: Azure Integration description: Azure OpenAI integration endpoints - name: Anthropic Integration description: Anthropic-compatible API endpoints (/anthropic/*) - name: GenAI Integration description: Google GenAI (Gemini) compatible API endpoints (/genai/*) - name: Bedrock Integration description: AWS Bedrock compatible API endpoints (/bedrock/*) - name: Cohere Integration description: Cohere compatible API endpoints (/cohere/*) - name: Typesafe Integration description: Typesafe compatible API endpoints (/typesafe/*) - name: LiteLLM Integration description: LiteLLM proxy endpoints with multi-provider support (/litellm/*) - name: LangChain Integration description: LangChain compatible endpoints with multi-provider support (/langchain/*) - name: PydanticAI Integration description: >- PydanticAI compatible endpoints with multi-provider support (/pydanticai/*) - name: Health description: Health check endpoints - name: Configuration description: Configuration management endpoints - name: Session description: Session and authentication endpoints - name: Providers description: Provider management endpoints - name: Plugins description: Plugin management endpoints - name: MCP description: Model Context Protocol endpoints - name: Governance description: Virtual keys, teams, and customers management - name: Routing description: Routing rules and complexity analyzer configuration - name: Logging description: Log search and management endpoints - name: Cache description: Cache management endpoints - name: Vault description: Vault secret management endpoints - name: Skills description: Skills Repository management, marketplace, and download endpoints - name: Audit Logs description: >- CADF-compliant audit log search, export, and signature verification endpoints - name: Webhooks description: Webhook endpoint management and signed async-job delivery history - name: Notifications description: >- Role-targeted dashboard notifications, delivered over the dashboard WebSocket - name: News Feeds description: >- RSS and Atom news sources shown in the News tab of the dashboard notification tray - name: Background Jobs description: >- Status, progress and cancellation of durable background jobs (cost recalculation, Warp log indexing, and others) - name: Warp description: >- Configuration for Warp, the dashboard agent that answers questions about the deployment's own telemetry - name: Relays description: >- Relays to non-LLM HTTP APIs - endpoint management, usage preview, Relay Logs and the /relay data plane paths: /typesafe/v1/systemone: post: tags: - Typesafe Integration summary: Evaluate state (Typesafe format) description: > Evaluates state against a map of named questions using Typesafe's native System One API. The request body matches Typesafe's native format; question identifiers become result field names. Images are listed in `assets` and placed by reference; send them base64 in JSON, or as file parts of a multipart request whose `request` part holds the JSON. Bifrost adds the `TypeSafe-Danger-Zone-Images` header Typesafe requires for images and box questions. operationId: typesafeSystemOne requestBody: required: true content: application/json: schema: type: object required: - state - model - questions properties: state: description: >- A plain string, or structured data (object or array) to evaluate. oneOf: - type: string - type: object additionalProperties: true - type: array items: {} model: type: string description: Typesafe model identifier (e.g. `jev-1.14.0`, `jev-latest`). assets: type: object description: >- Images keyed by the reference string written wherever each belongs in the state, instructions, or criteria. additionalProperties: type: object properties: type: type: string enum: - image format: type: string enum: - png - jpg - jpeg description: Required with `data_base64` data_base64: type: string description: >- The image file, base64-encoded, without a `data:` prefix part: type: string description: >- In a multipart request, the name of the file part holding the image questions: type: object minProperties: 1 description: Map of named questions. additionalProperties: type: object required: - type - instructions properties: type: type: string enum: - noul - choice - score - box description: >- Question type. `noul` returns a probability from 0 to 1, `choice` a selection with probabilities, `score` a probability-weighted rubric value, `box` the edges of a box around the instructions' subject in an image (Jev 1.14). instructions: description: Question instructions. A string, or structured data. oneOf: - type: string - type: object additionalProperties: true - type: array items: {} criteria: description: > Optional `true`/`false` descriptions for `noul`; a map of option to description for `choice` (at most 255 options); an ordered array of 2-10 level descriptions for `score`. oneOf: - type: object additionalProperties: type: string - type: array items: type: string multipart/form-data: schema: type: object required: - request properties: request: type: string description: >- The JSON request, as for `application/json`, with each multipart asset's `part` naming a file part of this request. additionalProperties: type: string format: binary description: An image file part, named by exactly one asset's `part` responses: '200': description: Successful response in Typesafe's native shape content: application/json: schema: type: object required: - model - answers properties: model: type: string description: Model version that performed the evaluation answers: type: object description: Map matching question identifiers to answers additionalProperties: type: object required: - type properties: type: type: string enum: - noul - choice - score - box noul: type: number description: Probability from 0 to 1 (noul answers) choice: type: string description: Selected option (choice answers) score: type: number description: Probability-weighted value (score answers) box: type: object description: >- Box answers. Edges as fractions from 0 to 1 of the image, from its top-left required: - top - bottom - left - right properties: top: type: number bottom: type: number left: type: number right: type: number probabilities: type: object additionalProperties: type: number legend: type: object additionalProperties: type: string confidence: type: number usage: type: object properties: input_tokens: type: integer output_tokens: type: integer '400': description: Bad request content: application/json: schema: $ref: '#/components/schemas/BifrostError' '500': description: Internal server error content: application/json: schema: $ref: '#/components/schemas/BifrostError' security: - BearerAuth: [] - VirtualKeyAuth: [] - ApiKeyAuth: [] components: schemas: BifrostError: type: object description: Error response from Bifrost properties: event_id: type: string type: type: string is_bifrost_error: type: boolean status_code: type: integer error: $ref: '#/components/schemas/ErrorField' extra_fields: $ref: '#/components/schemas/BifrostErrorExtraFields' ErrorField: type: object properties: type: type: string code: type: string message: type: string param: type: string event_id: type: string BifrostErrorExtraFields: type: object properties: routing_info: $ref: '#/components/schemas/RoutingInfo' provider: $ref: '#/components/schemas/ModelProvider' deprecated: true description: 'Deprecated: use routing_info.provider.' original_model_requested: type: string deprecated: true description: >- Deprecated: use routing_info.primary_model when routing_info.is_fallback is true, otherwise routing_info.model. For the model the caller asked for, read routing_info.requested_model. resolved_model_used: type: string deprecated: true description: >- Deprecated: use routing_info.resolved_key_alias.model_id when an alias matched, otherwise routing_info.model. request_type: type: string mcp_request_type: type: string a2a_request_type: type: string description: Agent Gateway operation, for A2A errors a2a_agent_name: type: string error_type: type: string description: >- Normalized, low-cardinality classification of why the request failed, declared by whichever component refused it. Prefixed by fault domain (caller_, policy_, provider_, bifrost_). Absent on failures that were not classified at source. retry_after_ms: type: integer format: int64 minimum: 1000 maximum: 300000 description: >- The provider's hint for how long to wait before retrying, in milliseconds, read from its retry-after-ms or Retry-After header or its google.rpc.RetryInfo error detail, and clamped to between 1000 and 300000. Absent when the provider gave no explicit hint. raw_request: description: Raw provider request, when raw request capture is enabled raw_response: description: Raw provider response, when raw response capture is enabled converted_request_type: type: string dropped_compat_plugin_params: type: array items: type: string latency: type: integer format: int64 description: Latency in milliseconds key_statuses: type: array description: >- Per-key results, for requests that ran on every key such as listing models items: type: object properties: key_id: type: string status: type: string enum: - success - list_models_failed provider: $ref: '#/components/schemas/ModelProvider' error: $ref: '#/components/schemas/BifrostError' mcp_auth_required: type: object description: >- Set when a per-user MCP tool needs the caller to complete an inline auth flow properties: kind: type: string mcp_client_id: type: string mcp_client_name: type: string message: type: string authorize_url: type: string session_id: type: string subject_token_missing: type: boolean description: >- token_exchange only. True when there was nothing to exchange, including when the caller's Bifrost sign-in has expired; message says which exchange_error: type: string description: >- token_exchange only. The identity provider's error code and description when it refused the exchange billed_usage: $ref: '#/components/schemas/BifrostLLMUsage' description: >- Usage the provider reported and billed even though the request failed RoutingInfo: type: object description: >- Which provider, model and key handled the attempt that produced this response or error, and what the caller asked for. Also sent as x-bifrost-routing-info-* response headers. properties: provider: $ref: '#/components/schemas/ModelProvider' description: Provider that ran the attempt model: type: string description: Model sent to that provider key: type: string description: Name of the stored key used, never its ID or secret resolved_key_alias: $ref: '#/components/schemas/ResolvedKeyAlias' is_fallback: type: boolean description: >- True when a fallback served the request or returned the error. Omitted when false. primary_provider: $ref: '#/components/schemas/ModelProvider' description: The primary attempt's provider. Set only when a fallback ran. primary_model: type: string description: The primary attempt's model. Set only when a fallback ran. requested_provider: $ref: '#/components/schemas/ModelProvider' description: >- Provider the caller asked for, before routing rules, load balancing or session affinity changed it. Omitted when the caller sent a bare model. requested_model: type: string description: Model the caller asked for, before anything changed it server_side_fallback_model: type: string description: Model that answered when Anthropic switched models inside one call ModelProvider: type: string description: AI model provider identifier enum: - anthropic - azure - bedrock - bedrock_mantle - sagemaker - cerebras - cohere - deepseek - gemini - groq - mistral - ollama - opencode-go - opencode-zen - openai - parasail - perplexity - sgl - vertex - openrouter - elevenlabs - huggingface - nebius - xai - replicate - vllm - runway - runware - fireworks - sarvam - wafer - databricks - cloudflare - typesafe - minimax - byteplus BifrostLLMUsage: type: object description: Token usage information properties: prompt_tokens: type: integer description: > Total input tokens including any prompt-cache tokens (read + write). Subtract prompt_tokens_details.cached_read_tokens and prompt_tokens_details.cached_write_tokens to get the non-cached portion. prompt_tokens_details: $ref: '#/components/schemas/ChatPromptTokensDetails' completion_tokens: type: integer description: Number of output/completion tokens generated. completion_tokens_details: $ref: '#/components/schemas/ChatCompletionTokensDetails' total_tokens: type: integer tool_usage: type: object description: | Counts of provider-hosted (server-side) tool calls billed per call. properties: web_search: type: object properties: num_requests: type: integer description: Number of billable web search calls. cost: $ref: '#/components/schemas/BifrostCost' ResolvedKeyAlias: type: object description: Set only when the requested model matched a model alias on the key required: - model_id properties: model_id: type: string description: Model ID sent to the provider model_name: type: string description: Canonical model name, used for pricing and logs model_family: type: string description: Resolved model family ChatPromptTokensDetails: type: object properties: text_tokens: type: integer audio_tokens: type: integer image_tokens: type: integer cached_read_tokens: type: integer description: > Tokens served from the prompt cache (cache hit). These tokens are already included in prompt_tokens and are billed at the reduced cache-read rate. Populated for all providers that support prompt caching (Anthropic, Bedrock, OpenAI, Gemini, xAI, etc.). cached_write_tokens: type: integer description: > Tokens written to the prompt cache on this request (cache creation / write). These tokens are already included in prompt_tokens and are billed at the cache-creation rate. Populated for providers that separately report cache write tokens (Anthropic, Bedrock). ChatCompletionTokensDetails: type: object properties: text_tokens: type: integer accepted_prediction_tokens: type: integer audio_tokens: type: integer citation_tokens: type: integer num_search_queries: type: integer deprecated: true description: >- Deprecated, use tool_usage.web_search.num_requests. Will be removed in 3.0.0. reasoning_tokens: type: integer image_tokens: type: integer rejected_prediction_tokens: type: integer BifrostCost: type: object description: Cost breakdown for the request properties: input_tokens_cost: type: number output_tokens_cost: type: number reasoning_tokens_cost: type: number description: Cost for reasoning/thinking tokens (reasoning models) citation_tokens_cost: type: number description: Cost for citation tokens search_queries_cost: type: number description: Cost for web search queries request_cost: type: number total_cost: type: number securitySchemes: BearerAuth: type: http scheme: bearer description: > Bearer token authentication. Use your provider API key or Bifrost authentication token. Virtual keys (prefixed with `sk-bf-`) can also be passed here. BasicAuth: type: http scheme: basic description: > Basic authentication using the Bifrost admin username and password (`auth_config.admin_username` / `auth_config.admin_password`). Accepted on management APIs (`/api/*`, `/metrics`, `/ws`) only - the inference middleware never validates Basic credentials. ApiKeyAuth: type: apiKey in: header name: x-api-key description: | API key authentication via the `x-api-key` header. Virtual keys (prefixed with `sk-bf-`) can also be passed here. VirtualKeyAuth: type: apiKey in: header name: x-bf-vk description: > Bifrost Virtual Key for governance, routing, and access control. Supported on all inference endpoints (`/v1/*`, `/openai/*`, `/anthropic/*`, `/bedrock/*`, `/cohere/*`, `/genai/*`, `/langchain/*`, `/litellm/*`, `/pydanticai/*`, `/mcp`), not on management APIs (`/api/*`) - with the single exception of the self-service `GET /api/governance/virtual-keys/quota`, where the virtual key is the credential. Example: `sk-bf-*` prefixed keys. ```` This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.