Adding a new provider

Learn how to contribute a new provider to Bifrost.

This guide will walk you through creating a provider for Bifrost, testing locally, adding it to frontend and CI/CD.

Quick Reference: This guide uses simplified generic examples for clarity. For complete, production-ready implementations:

• OpenAI-compatible providers: See core/providers/cerebras/ or core/providers/groq/
• Custom API providers: See core/providers/huggingface/ or core/providers/anthropic/

Setup

1. Fork and Clone:
   • Fork the repository: https://github.com/maximhq/bifrost/
   • Clone your fork: git clone https://github.com/<your_github_username>/bifrost/
2. Initialize:
   • Run make dev at the root of the project to set up dependencies and tools.

Provider Structure

Bifrost acts as a gateway:

1. Receives a request in a standard format (defined in core/schemas/).
2. Converts it to the provider-specific format.
3. Sends the request to the provider’s API.
4. Receives the provider’s response.
5. Converts it back to the standard Bifrost response format.

To implement a new provider, first step is to add the provider name in core/schemas/bifrost.go

• Add it in const declaration of ModelProvider type in the format [ProviderName] ModelProvider = "[providername]"
• Then, add it in the StandardProviders array in the same file and if needed add in SupportedBaseProviders.

Next, you will create a directory in core/providers/ and populate it with specific files following our strict conventions.

Directory Structure

The directory structure differs based on whether the provider is OpenAI API compatible:

Non-OpenAI-compatible Providers

If the provider has a custom API format (not OpenAI-compatible), create a new folder core/providers/[provider_name]/.

Complete Reference Structure (see core/providers/huggingface/):

core/providers/
└─ [provider_name]/                 # e.g., huggingface/
    ├── [provider_name].go          # Main provider implementation (REQUIRED)
    ├── [provider_name]_test.go     # Provider automated tests (REQUIRED)
    ├── types.go                    # ALL provider-specific types/structs (REQUIRED)
    ├── utils.go                    # ALL utility functions and constants (REQUIRED)
    ├── errors.go                   # Error handling (if supported)
    ├── chat.go                     # Converters for Chat Completion (if supported)
    ├── speech.go                   # Converters for Text-to-Speech (if supported)
    ├── transcription.go            # Converters for Speech-to-Text (if supported)
    ├── embedding.go                # Converters for Embeddings (if supported)
    ├── images.go                   # Converters for Images (if supported)
    ├── batches.go                  # Converters for Batches (if supported)
    ├── files.go                    # Converters for Files (if supported)
    ├── models.go                   # Converters for List Models (if supported)
    └── responses.go                # Converters for Response Models (if supported)

File Creation Order (CRITICAL):

1. Create types.go FIRST - Define all provider-specific request/response structures
2. Create utils.go SECOND - Define constants, base URLs, and helper functions
3. Create feature files (chat.go, embedding.go, etc.) THIRD - Implement converters
4. Create [provider_name].go FOURTH - Wire everything together
5. Create [provider_name]_test.go LAST - Add comprehensive tests

OpenAI-compatible Providers

If the provider is OpenAI API compatible, you only need a minimal structure:

Minimal Reference Structure (see core/providers/cerebras/):

core/providers/
└─ [provider_name]/                 # e.g., cerebras/
    ├── [provider_name].go          # Main provider implementation (REQUIRED)
    └── [provider_name]_test.go     # Provider automated tests (REQUIRED)

These providers reuse the OpenAI converter logic from core/providers/openai/.

File Conventions & Responsibilities

We enforce strict separation of concerns to keep providers maintainable and consistent. Each file has a specific purpose and must follow these rules.

---

1. types.go (The Data Layer)

CRITICAL RULE: All provider-specific structs (Request/Response DTOs) MUST go here. NEVER define types in other files.

Naming Convention:

• Prefix ALL types with the provider name in PascalCase: [ProviderName][StructName]
• Examples: HuggingFaceChatRequest, HuggingFaceModel, HuggingFaceToolCall

JSON Tag Requirements:

• Use json tags that exactly match the provider’s API field names
• Use omitempty for optional fields
• Use pointers for nullable fields to distinguish between “not set” and “zero value”

Organization:

• Group related types together with comments (e.g., // # CHAT TYPES, // # MODELS TYPES)
• Define request types before response types
• Keep nested types near their parent types

Generic Example Structure:

package providername

import "encoding/json"

// # MODELS TYPES

// ProviderNameModel represents a model from the provider's catalog
type ProviderNameModel struct {
    ID          string   `json:"id"`
    Name        string   `json:"name"`
    Description *string  `json:"description,omitempty"`
    CreatedAt   string   `json:"created_at"`
}

// # CHAT TYPES

// ProviderNameChatRequest represents the request payload for chat completion
type ProviderNameChatRequest struct {
    Model       string                      `json:"model" validate:"required"`
    Messages    []ProviderNameChatMessage   `json:"messages"`
    MaxTokens   *int                        `json:"max_tokens,omitempty"`
    Temperature *float64                    `json:"temperature,omitempty"`
    TopP        *float64                    `json:"top_p,omitempty"`
    Stream      *bool                       `json:"stream,omitempty"`
    Tools       []ProviderNameTool          `json:"tools,omitempty"`
    ToolChoice  json.RawMessage             `json:"tool_choice,omitempty"` // flexible: enum or object
}

// ProviderNameChatMessage represents a single message in a chat
type ProviderNameChatMessage struct {
    Role       *string         `json:"role,omitempty"`
    Content    json.RawMessage `json:"content,omitempty"` // flexible: string or []content items
    Name       *string         `json:"name,omitempty"`
    ToolCalls  []ProviderNameToolCall `json:"tool_calls,omitempty"`
}

Key Points:

• Use json.RawMessage for fields that can be multiple types (string or object/array)
• Use pointers (*float64, *bool) for optional fields
• Add validation tags when appropriate (validate:"required")
• Include comments for complex or non-obvious types

---

2. utils.go (The Helper Layer)

CRITICAL RULE: All shared utility functions, constants, and configuration helpers MUST go here.

Constants Naming Convention:

• Use camelCase for unexported constants: defaultInferenceBaseURL
• Use SCREAMING_SNAKE_CASE for exported constants: INFERENCE_PROVIDERS
• Group related constants together

Function Naming Convention:

• Use camelCase for unexported helpers: convertTypeToLowerCase, parseErrorResponse
• Use PascalCase for exported utilities: ConfigureProxy, BuildHeaders

Required Contents:

1. Base URLs and API endpoints
2. Default values and limits
3. Provider-specific constants (like model names, inference providers)
4. HTTP request helpers (headers, authentication)
5. Error handling utilities
6. Data transformation helpers

Generic Example Structure:

package providername

import (
    "context"
    "strings"
    "time"

    providerUtils "github.com/maximhq/bifrost/core/providers/utils"
    schemas "github.com/maximhq/bifrost/core/schemas"
    "github.com/valyala/fasthttp"
)

const (
    defaultBaseURL = "https://api.provider.com"
)

const (
    defaultLimit = 100
    maxLimit     = 500
)

// Helper to parse provider-specific model format
func parseModelString(model string) (string, string) {
    parts := strings.Split(model, "/")
    if len(parts) == 2 {
        return parts[0], parts[1]
    }
    return "", model
}

// Helper to convert type fields to lowercase in JSON schemas
func convertTypeToLowerCase(schema map[string]interface{}) {
    // Implementation for schema normalization...
}

Organization Tips:

• Group constants by category (URLs, limits, enums)
• Document the source/reason for constants (API docs, limits)
• Keep helper functions focused and single-purpose
• Include error handling in utility functions

---

3. [provider_name].go (The Controller Layer)

CRITICAL RULE: This is the orchestration layer. It coordinates the request flow but delegates all conversion logic to feature files.

Naming Convention:

• Provider struct: [ProviderName]Provider (e.g., HuggingFaceProvider)
• Constructor: New[ProviderName]Provider(config *schemas.ProviderConfig, logger schemas.Logger)
• Methods: Match interface exactly: ChatCompletion, ChatCompletionStream, ListModels, etc.

Required Struct Fields (in order):

type [ProviderName]Provider struct {
    logger               schemas.Logger              // ALWAYS first
    client               *fasthttp.Client            // HTTP client
    networkConfig        schemas.NetworkConfig       // Network settings
    sendBackRawResponse  bool                        // Debug flag
    customProviderConfig *schemas.CustomProviderConfig // Optional
}

Constructor Requirements:

1. Accept *schemas.ProviderConfig and schemas.Logger
2. Call config.CheckAndSetDefaults()
3. Initialize fasthttp.Client with timeouts and limits
4. Configure proxy using providerUtils.ConfigureProxy
5. Set default BaseURL if not provided
6. Trim trailing slashes from BaseURL
7. Pre-warm response pools if using sync.Pool
8. Return provider instance (and error for OpenAI-compatible providers)

Generic Example Structure:

package providername

import (
    "context"
    "strings"
    "sync"
    "time"

    "github.com/bytedance/sonic"
    providerUtils "github.com/maximhq/bifrost/core/providers/utils"
    schemas "github.com/maximhq/bifrost/core/schemas"
    "github.com/valyala/fasthttp"
)

// ProviderNameProvider implements the Provider interface
type ProviderNameProvider struct {
    logger               schemas.Logger
    client               *fasthttp.Client
    networkConfig        schemas.NetworkConfig
    sendBackRawResponse  bool
    customProviderConfig *schemas.CustomProviderConfig
}

// Response pools for memory efficiency (optional but recommended)
var chatResponsePool = sync.Pool{
    New: func() any {
        return &ProviderNameChatResponse{}
    },
}

// NewProviderNameProvider creates a new provider instance
func NewProviderNameProvider(config *schemas.ProviderConfig, logger schemas.Logger) *ProviderNameProvider {
    config.CheckAndSetDefaults()

    client := &fasthttp.Client{
        ReadTimeout:         time.Second * time.Duration(config.NetworkConfig.DefaultRequestTimeoutInSeconds),
        WriteTimeout:        time.Second * time.Duration(config.NetworkConfig.DefaultRequestTimeoutInSeconds),
        MaxConnsPerHost:     5000,
        MaxIdleConnDuration: 30 * time.Second,
        MaxConnWaitTimeout:  10 * time.Second,
    }

    // Configure proxy if provided
    client = providerUtils.ConfigureProxy(client, config.ProxyConfig, logger)

    // Set default BaseURL if not provided
    if config.NetworkConfig.BaseURL == "" {
        config.NetworkConfig.BaseURL = defaultBaseURL
    }
    config.NetworkConfig.BaseURL = strings.TrimRight(config.NetworkConfig.BaseURL, "/")

    // Pre-warm response pools (optional optimization)
    for i := 0; i < config.ConcurrencyAndBufferSize.Concurrency; i++ {
        chatResponsePool.Put(&ProviderNameChatResponse{})
    }

    return &ProviderNameProvider{
        logger:               logger,
        client:               client,
        networkConfig:        config.NetworkConfig,
        sendBackRawResponse:  config.SendBackRawResponse,
        customProviderConfig: config.CustomProviderConfig,
    }
}

// GetProviderKey returns the provider identifier
func (provider *ProviderNameProvider) GetProviderKey() schemas.ModelProvider {
    return schemas.ProviderName
}

Method Implementation Pattern (STRICT ORDER):

1. Validation: Check request validity (optional, usually done in converter)
2. Convert Request: Call To[Provider][Feature]Request() from feature file
3. Build HTTP Request: Construct URL, headers, body
4. Execute Request: Use provider.client.Do() or streaming logic
5. Handle Errors: Parse and convert provider errors to schemas.BifrostError
6. Convert Response: Call ToBifrost[Feature]Response() from feature file
7. Return Result: Return Bifrost response or error

Example Method (generic pattern):

func (p *ProviderNameProvider) ChatCompletion(
    ctx context.Context,
    key schemas.Key,
    request *schemas.BifrostChatRequest,
) (*schemas.BifrostChatResponse, *schemas.BifrostError) {
    // 1. Convert Request
    providerReq := ToProviderNameChatCompletionRequest(request)
    
    // 2. Build HTTP Request
    body, err := sonic.Marshal(providerReq)
    if err != nil {
        return nil, &schemas.BifrostError{/* ... */}
    }
    
    req := fasthttp.AcquireRequest()
    defer fasthttp.ReleaseRequest(req)
    
    req.SetRequestURI(p.networkConfig.BaseURL + "/v1/chat/completions")
    req.Header.SetMethod("POST")
    req.Header.Set("Authorization", "Bearer "+key.Value)
    req.Header.Set("Content-Type", "application/json")
    req.SetBody(body)
    
    // 3. Execute Request
    resp := fasthttp.AcquireResponse()
    defer fasthttp.ReleaseResponse(resp)
    
    if err := p.client.Do(req, resp); err != nil {
        return nil, &schemas.BifrostError{/* ... */}
    }
    
    // 4. Handle Errors
    if resp.StatusCode() != 200 {
        return nil, parseErrorResponse(resp.Body())
    }
    
    // 5. Convert Response
    var providerResp ProviderNameChatResponse
    if err := sonic.Unmarshal(resp.Body(), &providerResp); err != nil {
        return nil, &schemas.BifrostError{/* ... */}
    }
    
    return ToBifrostChatResponse(&providerResp)
}

---

4. Feature Files (chat.go, embedding.go, speech.go, etc.) (The Converter Layer)

CRITICAL RULE: These files contain pure transformation functions ONLY. No HTTP calls, no logging, no side effects.

File Naming Convention:

• chat.go - Chat completion converters
• embedding.go - Embedding converters
• speech.go - Text-to-speech converters
• transcription.go - Speech-to-text converters
• models.go - List models converters
• responses.go - Response format converters

Function Naming Convention (STRICT):

• To Provider Format: To[ProviderName][Feature]Request(bifrostReq *schemas.Bifrost[Feature]Request) *[ProviderName][Feature]Request
• To Bifrost Format: ToBifrost[Feature]Response(providerResp *[ProviderName][Feature]Response) (*schemas.Bifrost[Feature]Response, *schemas.BifrostError)

Examples:

• ToHuggingFaceChatCompletionRequest
• ToBifrostChatResponse
• ToHuggingFaceEmbeddingRequest
• ToBifrostEmbeddingResponse

Required Converter Pairs (if feature is supported):

• Request converter: Bifrost → Provider
• Response converter: Provider → Bifrost

Real Example from core/providers/huggingface/chat.go:

package huggingface

import (
    "encoding/json"
    "fmt"

    "github.com/bytedance/sonic"
    schemas "github.com/maximhq/bifrost/core/schemas"
)

// ToHuggingFaceChatCompletionRequest converts a Bifrost chat request to HuggingFace format
func ToHuggingFaceChatCompletionRequest(bifrostReq *schemas.BifrostChatRequest) *HuggingFaceChatRequest {
    if bifrostReq == nil || bifrostReq.Input == nil {
        return nil
    }

    // Convert messages from Bifrost format to HuggingFace format
    hfMessages := make([]HuggingFaceChatMessage, 0, len(bifrostReq.Input))
    for _, msg := range bifrostReq.Input {
        hfMsg := HuggingFaceChatMessage{}

        // Set role
        if msg.Role != "" {
            role := string(msg.Role)
            hfMsg.Role = &role
        }

        // Set name if present
        if msg.Name != nil {
            hfMsg.Name = msg.Name
        }

        // Convert content (can be string or structured blocks)
        if msg.Content != nil {
            if msg.Content.ContentStr != nil {
                // Simple string content
                contentJSON, _ := sonic.Marshal(*msg.Content.ContentStr)
                hfMsg.Content = json.RawMessage(contentJSON)
            } else if msg.Content.ContentBlocks != nil {
                // Structured content blocks (text, images, etc.)
                contentItems := make([]HuggingFaceContentItem, 0, len(msg.Content.ContentBlocks))
                for _, block := range msg.Content.ContentBlocks {
                    item := HuggingFaceContentItem{}
                    blockType := string(block.Type)
                    item.Type = &blockType

                    switch block.Type {
                    case schemas.ChatContentBlockTypeText:
                        if block.Text != nil {
                            item.Text = block.Text
                        }
                    case schemas.ChatContentBlockTypeImage:
                        if block.ImageURLStruct != nil {
                            item.ImageURL = &HuggingFaceImageRef{
                                URL: block.ImageURLStruct.URL,
                            }
                        }
                    }
                    contentItems = append(contentItems, item)
                }
                contentJSON, _ := sonic.Marshal(contentItems)
                hfMsg.Content = json.RawMessage(contentJSON)
            }
        }

        // Handle tool calls for assistant messages
        if msg.ChatAssistantMessage != nil && len(msg.ChatAssistantMessage.ToolCalls) > 0 {
            hfToolCalls := make([]HuggingFaceToolCall, 0, len(msg.ChatAssistantMessage.ToolCalls))
            for _, tc := range msg.ChatAssistantMessage.ToolCalls {
                hfToolCall := HuggingFaceToolCall{
                    ID:   tc.ID,
                    Type: tc.Type,
                    Function: HuggingFaceFunction{
                        Name:      *tc.Function.Name,
                        Arguments: tc.Function.Arguments,
                    },
                }
                hfToolCalls = append(hfToolCalls, hfToolCall)
            }
            hfMsg.ToolCalls = hfToolCalls
        }

        hfMessages = append(hfMessages, hfMsg)
    }

    // Build the request
    hfReq := &HuggingFaceChatRequest{
        Model:    bifrostReq.Model,
        Messages: hfMessages,
    }

    // Map parameters
    if bifrostReq.Params != nil {
        params := bifrostReq.Params
        
        // Map standard parameters
        if params.Temperature != nil {
            hfReq.Temperature = params.Temperature
        }
        if params.MaxTokens != nil {
            hfReq.MaxTokens = params.MaxTokens
        }
        // ... other standard parameters
        
        // Handle provider-specific ExtraParams
        if params.ExtraParams != nil {
            if customParam, ok := params.ExtraParams["custom_param"].(string); ok {
                hfReq.CustomParam = &customParam
            }
        }
    }

    return hfReq
}

Generic Example - Embedding Converter:

package providername

import (
    "github.com/maximhq/bifrost/core/schemas"
)

// ToProviderNameEmbeddingRequest converts a Bifrost embedding request to provider format
func ToProviderNameEmbeddingRequest(bifrostReq *schemas.BifrostEmbeddingRequest) *ProviderNameEmbeddingRequest {
    if bifrostReq == nil {
        return nil
    }

    providerReq := &ProviderNameEmbeddingRequest{
        Model: bifrostReq.Model,
    }

    // Convert input
    if bifrostReq.Input != nil {
        if bifrostReq.Input.Text != nil {
            providerReq.Input = *bifrostReq.Input.Text
        } else if bifrostReq.Input.Texts != nil {
            providerReq.Input = bifrostReq.Input.Texts
        }
    }

    // Map provider-specific parameters from ExtraParams
    if bifrostReq.Params != nil && bifrostReq.Params.ExtraParams != nil {
        if normalize, ok := bifrostReq.Params.ExtraParams["normalize"].(bool); ok {
            providerReq.Normalize = &normalize
        }
    }

    return providerReq
}

Generic Example - List Models Converter:

package providername

import (
    "strings"
    schemas "github.com/maximhq/bifrost/core/schemas"
)

// ToBifrostListModelsResponse converts provider models list to Bifrost format
func ToBifrostListModelsResponse(
    providerResp *ProviderNameListModelsResponse,
    providerKey schemas.ModelProvider,
) *schemas.BifrostListModelsResponse {
    if providerResp == nil {
        return nil
    }

    bifrostResponse := &schemas.BifrostListModelsResponse{
        Data: make([]schemas.Model, 0, len(providerResp.Models)),
    }
    
    for _, model := range providerResp.Models {
        // Determine supported methods based on model capabilities
        supported := determineSupportedMethods(model)
        if len(supported) == 0 {
            continue
        }

        newModel := schemas.Model{
            ID:               model.ID,
            Name:             &model.Name,
            SupportedMethods: supported,
        }

        bifrostResponse.Data = append(bifrostResponse.Data, newModel)
    }
    
    return bifrostResponse
}

// Helper to determine which Bifrost methods a model supports
func determineSupportedMethods(model ProviderNameModel) []string {
    methods := []string{}
    
    // Logic to derive supported methods from model metadata
    // This varies by provider
    
    return methods
}

Converter Best Practices:

1. Always check for nil inputs at the start
2. Pre-allocate slices with known capacity for performance
3. Handle optional fields using pointers in types
4. Use ExtraParams for provider-specific fields not in standard schema
5. Document complex conversions with inline comments
6. Keep functions pure - no side effects, no external state
7. Return errors when conversion fails (for response converters)

OpenAI-compatible Providers

If you are implementing a provider that is strictly OpenAI API compatible, the implementation is significantly simpler. You reuse all the conversion logic from core/providers/openai/.

When to Use This Approach:

• Provider’s API is 100% OpenAI-compatible
• Same request/response formats
• Same endpoint paths (/v1/chat/completions, /v1/completions, etc.)
• Only differences are: base URL, authentication, and possibly some extra headers

Complete Reference: core/providers/cerebras/cerebras.go

---

Step 1: Create the Provider File

Create core/providers/[provider_name]/[provider_name].go:

// Package cerebras implements the Cerebras LLM provider.
package cerebras

import (
    "context"
    "strings"
    "time"

    "github.com/maximhq/bifrost/core/providers/openai"
    providerUtils "github.com/maximhq/bifrost/core/providers/utils"
    schemas "github.com/maximhq/bifrost/core/schemas"
    "github.com/valyala/fasthttp"
)

// CerebrasProvider implements the Provider interface for Cerebras's API.
type CerebrasProvider struct {
    logger              schemas.Logger        // Logger for provider operations
    client              *fasthttp.Client      // HTTP client for API requests
    networkConfig       schemas.NetworkConfig // Network configuration including extra headers
    sendBackRawResponse bool                  // Whether to include raw response in BifrostResponse
}

// NewCerebrasProvider creates a new Cerebras provider instance.
// It initializes the HTTP client with the provided configuration and sets up response pools.
func NewCerebrasProvider(config *schemas.ProviderConfig, logger schemas.Logger) (*CerebrasProvider, error) {
    config.CheckAndSetDefaults()

    client := &fasthttp.Client{
        ReadTimeout:         time.Second * time.Duration(config.NetworkConfig.DefaultRequestTimeoutInSeconds),
        WriteTimeout:        time.Second * time.Duration(config.NetworkConfig.DefaultRequestTimeoutInSeconds),
        MaxConnsPerHost:     5000,
        MaxIdleConnDuration: 30 * time.Second,
        MaxConnWaitTimeout:  10 * time.Second,
    }

    // Configure proxy if provided
    client = providerUtils.ConfigureProxy(client, config.ProxyConfig, logger)

    // Set default BaseURL if not provided
    if config.NetworkConfig.BaseURL == "" {
        config.NetworkConfig.BaseURL = "https://api.cerebras.ai"
    }
    config.NetworkConfig.BaseURL = strings.TrimRight(config.NetworkConfig.BaseURL, "/")

    return &CerebrasProvider{
        logger:              logger,
        client:              client,
        networkConfig:       config.NetworkConfig,
        sendBackRawResponse: config.SendBackRawResponse,
    }, nil
}

// GetProviderKey returns the provider identifier for Cerebras.
func (provider *CerebrasProvider) GetProviderKey() schemas.ModelProvider {
    return schemas.Cerebras
}

---

Step 2: Implement Required Methods Using OpenAI Handlers

For each supported feature, delegate to the corresponding OpenAI handler:

Chat Completion (Non-Streaming):

// ChatCompletion performs a chat completion request to the Cerebras API.
func (provider *CerebrasProvider) ChatCompletion(
    ctx context.Context,
    key schemas.Key,
    request *schemas.BifrostChatRequest,
) (*schemas.BifrostChatResponse, *schemas.BifrostError) {
    return openai.HandleOpenAIChatCompletionRequest(
        ctx,
        provider.client,
        provider.networkConfig.BaseURL+providerUtils.GetPathFromContext(ctx, "/v1/chat/completions"),
        request,
        key,
        provider.networkConfig.ExtraHeaders,
        providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse),
        provider.GetProviderKey(),
        provider.logger,
    )
}

Chat Completion (Streaming):

// ChatCompletionStream performs a streaming chat completion request to the Cerebras API.
// It supports real-time streaming of responses using Server-Sent Events (SSE).
func (provider *CerebrasProvider) ChatCompletionStream(
    ctx context.Context,
    postHookRunner schemas.PostHookRunner,
    key schemas.Key,
    request *schemas.BifrostChatRequest,
) (chan *schemas.BifrostStream, *schemas.BifrostError) {
    var authHeader map[string]string
    if key.Value != "" {
        authHeader = map[string]string{"Authorization": "Bearer " + key.Value}
    }
    
    // Use shared OpenAI-compatible streaming logic
    return openai.HandleOpenAIChatCompletionStreaming(
        ctx,
        provider.client,
        provider.networkConfig.BaseURL+"/v1/chat/completions",
        request,
        authHeader,
        provider.networkConfig.ExtraHeaders,
        providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse),
        provider.GetProviderKey(),
        postHookRunner,
        nil, // customStreamParser - use nil for standard OpenAI format
        provider.logger,
    )
}

Text Completion (Non-Streaming):

// TextCompletion performs a text completion request to Cerebras's API.
func (provider *CerebrasProvider) TextCompletion(
    ctx context.Context,
    key schemas.Key,
    request *schemas.BifrostTextCompletionRequest,
) (*schemas.BifrostTextCompletionResponse, *schemas.BifrostError) {
    return openai.HandleOpenAITextCompletionRequest(
        ctx,
        provider.client,
        provider.networkConfig.BaseURL+providerUtils.GetPathFromContext(ctx, "/v1/completions"),
        request,
        key,
        provider.networkConfig.ExtraHeaders,
        provider.GetProviderKey(),
        providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse),
        provider.logger,
    )
}

Text Completion (Streaming):

// TextCompletionStream performs a streaming text completion request to Cerebras's API.
func (provider *CerebrasProvider) TextCompletionStream(
    ctx context.Context,
    postHookRunner schemas.PostHookRunner,
    key schemas.Key,
    request *schemas.BifrostTextCompletionRequest,
) (chan *schemas.BifrostStream, *schemas.BifrostError) {
    var authHeader map[string]string
    if key.Value != "" {
        authHeader = map[string]string{"Authorization": "Bearer " + key.Value}
    }
    
    return openai.HandleOpenAITextCompletionStreaming(
        ctx,
        provider.client,
        provider.networkConfig.BaseURL+"/v1/completions",
        request,
        authHeader,
        provider.networkConfig.ExtraHeaders,
        providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse),
        provider.GetProviderKey(),
        postHookRunner,
        nil, // customStreamParser
        provider.logger,
    )
}

List Models:

// ListModels performs a list models request to Cerebras's API.
func (provider *CerebrasProvider) ListModels(
    ctx context.Context,
    keys []schemas.Key,
    request *schemas.BifrostListModelsRequest,
) (*schemas.BifrostListModelsResponse, *schemas.BifrostError) {
    return openai.HandleOpenAIListModelsRequest(
        ctx,
        provider.client,
        request,
        provider.networkConfig.BaseURL+providerUtils.GetPathFromContext(ctx, "/v1/models"),
        keys,
        provider.networkConfig.ExtraHeaders,
        provider.GetProviderKey(),
        providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse),
        provider.logger,
    )
}

---

Step 3: Implement Unsupported Methods

For features not supported by the provider, return appropriate errors:

// Embedding is not supported by Cerebras
func (provider *CerebrasProvider) Embedding(
    ctx context.Context,
    key schemas.Key,
    request *schemas.BifrostEmbeddingRequest,
) (*schemas.BifrostEmbeddingResponse, *schemas.BifrostError) {
    return nil, &schemas.BifrostError{
        StatusCode: http.StatusNotImplemented,
        Message:    "Embedding is not supported by Cerebras",
        Type:       "unsupported_feature",
    }
}

// Speech is not supported by Cerebras
func (provider *CerebrasProvider) Speech(
    ctx context.Context,
    key schemas.Key,
    request *schemas.BifrostSpeechRequest,
) (*schemas.BifrostSpeechResponse, *schemas.BifrostError) {
    return nil, &schemas.BifrostError{
        StatusCode: http.StatusNotImplemented,
        Message:    "Speech synthesis is not supported by Cerebras",
        Type:       "unsupported_feature",
    }
}

// SpeechStream is not supported by Cerebras
func (provider *CerebrasProvider) SpeechStream(
    ctx context.Context,
    postHookRunner schemas.PostHookRunner,
    key schemas.Key,
    request *schemas.BifrostSpeechRequest,
) (chan *schemas.BifrostStream, *schemas.BifrostError) {
    return nil, &schemas.BifrostError{
        StatusCode: http.StatusNotImplemented,
        Message:    "Speech synthesis streaming is not supported by Cerebras",
        Type:       "unsupported_feature",
    }
}

// Transcription is not supported by Cerebras
func (provider *CerebrasProvider) Transcription(
    ctx context.Context,
    key schemas.Key,
    request *schemas.BifrostTranscriptionRequest,
) (*schemas.BifrostTranscriptionResponse, *schemas.BifrostError) {
    return nil, &schemas.BifrostError{
        StatusCode: http.StatusNotImplemented,
        Message:    "Transcription is not supported by Cerebras",
        Type:       "unsupported_feature",
    }
}

// TranscriptionStream is not supported by Cerebras
func (provider *CerebrasProvider) TranscriptionStream(
    ctx context.Context,
    postHookRunner schemas.PostHookRunner,
    key schemas.Key,
    request *schemas.BifrostTranscriptionRequest,
) (chan *schemas.BifrostStream, *schemas.BifrostError) {
    return nil, &schemas.BifrostError{
        StatusCode: http.StatusNotImplemented,
        Message:    "Transcription streaming is not supported by Cerebras",
        Type:       "unsupported_feature",
    }
}

// Responses is not supported by Cerebras
func (provider *CerebrasProvider) Responses(
    ctx context.Context,
    key schemas.Key,
    request *schemas.BifrostResponsesRequest,
) (*schemas.BifrostResponsesResponse, *schemas.BifrostError) {
    return nil, &schemas.BifrostError{
        StatusCode: http.StatusNotImplemented,
        Message:    "Responses is not supported by Cerebras",
        Type:       "unsupported_feature",
    }
}

// ResponsesStream is not supported by Cerebras
func (provider *CerebrasProvider) ResponsesStream(
    ctx context.Context,
    postHookRunner schemas.PostHookRunner,
    key schemas.Key,
    request *schemas.BifrostResponsesRequest,
) (chan *schemas.BifrostStream, *schemas.BifrostError) {
    return nil, &schemas.BifrostError{
        StatusCode: http.StatusNotImplemented,
        Message:    "Responses streaming is not supported by Cerebras",
        Type:       "unsupported_feature",
    }
}

---

Key Points for OpenAI-compatible Providers

Constructor Differences:

• Returns (*[ProviderName]Provider, error) instead of just *[ProviderName]Provider
• Must set a default BaseURL specific to the provider
• Must trim trailing slashes from BaseURL

URL Construction:

• Use provider.networkConfig.BaseURL + "/v1/[endpoint]" for direct paths
• Use providerUtils.GetPathFromContext(ctx, "/v1/[endpoint]") when path might be overridden in context

Authentication Headers:

• Create authHeader map[string]string with Authorization: Bearer {key}
• Pass to OpenAI handlers separately from ExtraHeaders

Custom Stream Parsers:

• Pass nil for customStreamParser if using standard OpenAI SSE format
• Only implement custom parser if provider uses non-standard streaming format

Error Handling:

• OpenAI handlers return *schemas.BifrostError - propagate directly
• For unsupported features, return custom error with StatusNotImplemented

Advantages of This Approach:

• Automatic updates - benefits from OpenAI handler improvements
• Consistent behavior - same conversion logic as OpenAI
• Easy maintenance - only provider-specific config in your file

Implementation Steps

Follow this exact order when implementing a new provider.

---

For Non-OpenAI-compatible Providers

Phase 1: Research & Planning (Before Writing Code)

1. Study the Provider’s API Documentation:

   • Identify all supported endpoints (chat, embeddings, speech, etc.)
   • Note authentication method (API key, bearer token, custom headers)
   • Document base URL and endpoint paths
   • List all request/response fields
   • Identify provider-specific parameters not in OpenAI schema

2. Create a Mapping Document (recommended):

   # Provider: [ProviderName]
   
   ## Authentication
   - Method: Bearer token / API key in header
   - Header name: Authorization / X-API-Key
   
   ## Base URL
   - Production: https://api.provider.com
   - Staging: https://staging.provider.com (if applicable)
   
   ## Endpoints
   - Chat Completions: POST /v1/chat/completions
   - Embeddings: POST /v1/embeddings
   - Models: GET /v1/models
   
   ## Request Fields
   ### Chat Completions
   - model (required): string
   - messages (required): array
   - temperature (optional): float
   - max_tokens (optional): int
   - [provider_specific_field] (optional): type
   
   ## Response Fields
   ### Chat Completions
   - id: string
   - choices: array
   - usage: object
   - [provider_specific_field]: type
   

---

Phase 2: Create Directory Structure

3. Create Provider Directory:
   mkdir -p core/providers/[provider_name]
   cd core/providers/[provider_name]
   

---

Phase 3: Define Types (types.go)

4. Create types.go - Define ALL Provider-Specific Types:

   Order of Type Definitions:

   package [provider_name]
   
   import "encoding/json"
   
   // # MODELS TYPES
   // Define model-related types first
   type [ProviderName]Model struct { ... }
   type [ProviderName]ListModelsResponse struct { ... }
   
   // # CHAT TYPES
   // Define chat-related types
   type [ProviderName]ChatRequest struct { ... }
   type [ProviderName]ChatResponse struct { ... }
   type [ProviderName]ChatMessage struct { ... }
   type [ProviderName]ChatChoice struct { ... }
   
   // # EMBEDDING TYPES
   // Define embedding-related types
   type [ProviderName]EmbeddingRequest struct { ... }
   type [ProviderName]EmbeddingResponse struct { ... }
   
   // # SPEECH TYPES (if applicable)
   // Define speech-related types
   
   // # TRANSCRIPTION TYPES (if applicable)
   // Define transcription-related types
   
   // # ERROR TYPES
   // Define error response types
   type [ProviderName]ErrorResponse struct { ... }
   

   Type Naming Checklist:

   • ✅ All types prefixed with provider name: HuggingFaceChatRequest
   • ✅ JSON tags match provider API exactly: json:"model_name"
   • ✅ Optional fields use omitempty: json:"temperature,omitempty"
   • ✅ Nullable fields use pointers: *float64, *string
   • ✅ Flexible fields use json.RawMessage: Content json.RawMessage
   • ✅ Required fields have validation tags: validate:"required"

---

Phase 4: Define Utilities (utils.go)

5. Create utils.go - Define Constants and Helper Functions:

   Order of Definitions:

   package [provider_name]
   
   import (
       "context"
       "encoding/json"
       "fmt"
       
       providerUtils "github.com/maximhq/bifrost/core/providers/utils"
       schemas "github.com/maximhq/bifrost/core/schemas"
       "github.com/valyala/fasthttp"
   )
   
   // 1. BASE URLs (ALWAYS FIRST)
   const (
       defaultBaseURL = "https://api.provider.com"
       alternateURL   = "https://alternate.provider.com"
   )
   
   // 2. DEFAULT VALUES AND LIMITS
   const (
       defaultTimeout      = 60
       maxRequestSize      = 1024 * 1024 * 10 // 10MB
       defaultModelLimit   = 100
       maxConcurrentCalls  = 5000
   )
   
   // 3. PROVIDER-SPECIFIC ENUMS/CONSTANTS
   const (
       providerVersion = "v1"
       apiVersion      = "2024-01"
   )
   
   // 4. CUSTOM TYPES FOR CONSTANTS (if needed)
   type inferenceProvider string
   
   const (
       providerA inferenceProvider = "provider-a"
       providerB inferenceProvider = "provider-b"
   )
   
   // 5. HELPER FUNCTIONS
   // Function to build authentication headers
   func buildAuthHeaders(apiKey string) map[string]string { ... }
   
   // Function to parse error responses
   func parseErrorResponse(body []byte) *schemas.BifrostError { ... }
   
   // Function to validate model names
   func validateModelName(model string) error { ... }
   
   // Function to split composite model identifiers
   func splitModelProvider(model string) (provider, modelName string) { ... }
   

   Utility Function Checklist:

   • ✅ All base URLs defined as constants
   • ✅ Helper functions use camelCase (unexported) or PascalCase (exported)
   • ✅ Error handling utilities included
   • ✅ HTTP header builders included
   • ✅ Constants grouped logically with comments

---

Phase 5: Implement Converters (Feature Files)

6. Create Feature Files in Order of Complexity (simplest first):

   a. Create models.go (if supported):

   package [provider_name]
   
   import (
       "fmt"
       schemas "github.com/maximhq/bifrost/core/schemas"
   )
   
   // ToBifrostListModelsResponse converts provider models to Bifrost format
   func (response *[ProviderName]ListModelsResponse) ToBifrostListModelsResponse(
       providerKey schemas.ModelProvider,
   ) *schemas.BifrostListModelsResponse {
       if response == nil {
           return nil
       }
   
       bifrostResponse := &schemas.BifrostListModelsResponse{
           Data: make([]schemas.Model, 0, len(response.Models)),
       }
   
       for _, model := range response.Models {
           // Validation
           if model.ID == "" {
               continue
           }
   
           // Conversion logic
           bifrostModel := schemas.Model{
               ID:               fmt.Sprintf("%s/%s", providerKey, model.ID),
               Name:             &model.Name,
               SupportedMethods: deriveSupportedMethods(model),
           }
   
           bifrostResponse.Data = append(bifrostResponse.Data, bifrostModel)
       }
   
       return bifrostResponse
   }
   
   // Helper function to determine supported methods
   func deriveSupportedMethods(model [ProviderName]Model) []string {
       // Implementation
   }
   

   b. Create embedding.go (if supported):

   package [provider_name]
   
   import schemas "github.com/maximhq/bifrost/core/schemas"
   
   // To[ProviderName]EmbeddingRequest converts Bifrost request to provider format
   func To[ProviderName]EmbeddingRequest(
       bifrostReq *schemas.BifrostEmbeddingRequest,
   ) *[ProviderName]EmbeddingRequest {
       if bifrostReq == nil {
           return nil
       }
   
       providerReq := &[ProviderName]EmbeddingRequest{
           Model: bifrostReq.Model,
       }
   
       // Convert input
       if bifrostReq.Input != nil {
           if bifrostReq.Input.Text != nil {
               providerReq.Input = *bifrostReq.Input.Text
           } else if bifrostReq.Input.Texts != nil {
               providerReq.Input = bifrostReq.Input.Texts
           }
       }
   
       // Map parameters
       if bifrostReq.Params != nil {
           // Standard parameters
           if bifrostReq.Params.Dimensions != nil {
               providerReq.Dimensions = bifrostReq.Params.Dimensions
           }
   
           // Provider-specific parameters from ExtraParams
           if bifrostReq.Params.ExtraParams != nil {
               if val, ok := bifrostReq.Params.ExtraParams["provider_param"].(string); ok {
                   providerReq.ProviderParam = &val
               }
           }
       }
   
       return providerReq
   }
   
   // ToBifrostEmbeddingResponse converts provider response to Bifrost format
   func ToBifrostEmbeddingResponse(
       providerResp *[ProviderName]EmbeddingResponse,
   ) (*schemas.BifrostEmbeddingResponse, *schemas.BifrostError) {
       if providerResp == nil {
           return nil, &schemas.BifrostError{
               Message: "Provider response is nil",
               Type:    "invalid_response",
           }
       }
   
       bifrostResp := &schemas.BifrostEmbeddingResponse{
           Data: make([]schemas.EmbeddingData, 0, len(providerResp.Data)),
       }
   
       for i, embedding := range providerResp.Data {
           bifrostResp.Data = append(bifrostResp.Data, schemas.EmbeddingData{
               Index:     i,
               Embedding: embedding.Values,
           })
       }
   
       // Map usage if available
       if providerResp.Usage != nil {
           bifrostResp.Usage = &schemas.Usage{
               PromptTokens: providerResp.Usage.InputTokens,
               TotalTokens:  providerResp.Usage.TotalTokens,
           }
       }
   
       return bifrostResp, nil
   }
   

   c. Create chat.go (most complex):

   package [provider_name]
   
   import (
       "encoding/json"
       "github.com/bytedance/sonic"
       schemas "github.com/maximhq/bifrost/core/schemas"
   )
   
   // To[ProviderName]ChatCompletionRequest converts Bifrost chat request to provider format
   func To[ProviderName]ChatCompletionRequest(
       bifrostReq *schemas.BifrostChatRequest,
   ) *[ProviderName]ChatRequest {
       if bifrostReq == nil || bifrostReq.Input == nil {
           return nil
       }
   
       // Convert messages
       providerMessages := make([][ProviderName]ChatMessage, 0, len(bifrostReq.Input))
       for _, msg := range bifrostReq.Input {
           providerMsg := [ProviderName]ChatMessage{}
   
           // Set role
           if msg.Role != "" {
               role := string(msg.Role)
               providerMsg.Role = &role
           }
   
           // Set name if present
           if msg.Name != nil {
               providerMsg.Name = msg.Name
           }
   
           // Convert content (can be string or structured)
           if msg.Content != nil {
               if msg.Content.ContentStr != nil {
                   // Simple string content
                   contentJSON, _ := sonic.Marshal(*msg.Content.ContentStr)
                   providerMsg.Content = json.RawMessage(contentJSON)
               } else if msg.Content.ContentBlocks != nil {
                   // Structured content (text, images, etc.)
                   contentItems := make([][ProviderName]ContentItem, 0, len(msg.Content.ContentBlocks))
                   for _, block := range msg.Content.ContentBlocks {
                       item := [ProviderName]ContentItem{}
                       blockType := string(block.Type)
                       item.Type = &blockType
   
                       switch block.Type {
                       case schemas.ChatContentBlockTypeText:
                           if block.Text != nil {
                               item.Text = block.Text
                           }
                       case schemas.ChatContentBlockTypeImage:
                           if block.ImageURLStruct != nil {
                               item.ImageURL = &[ProviderName]ImageRef{
                                   URL: block.ImageURLStruct.URL,
                               }
                           }
                       }
                       contentItems = append(contentItems, item)
                   }
                   contentJSON, _ := sonic.Marshal(contentItems)
                   providerMsg.Content = json.RawMessage(contentJSON)
               }
           }
   
           // Handle tool calls for assistant messages
           if msg.ChatAssistantMessage != nil && len(msg.ChatAssistantMessage.ToolCalls) > 0 {
               providerToolCalls := make([][ProviderName]ToolCall, 0, len(msg.ChatAssistantMessage.ToolCalls))
               for _, tc := range msg.ChatAssistantMessage.ToolCalls {
                   providerToolCall := [ProviderName]ToolCall{
                       ID:   tc.ID,
                       Type: tc.Type,
                       Function: [ProviderName]Function{
                           Name:      *tc.Function.Name,
                           Arguments: tc.Function.Arguments,
                       },
                   }
                   providerToolCalls = append(providerToolCalls, providerToolCall)
               }
               providerMsg.ToolCalls = providerToolCalls
           }
   
           // Handle tool call responses
           if msg.ChatToolMessage != nil && msg.ChatToolMessage.ToolCallID != nil {
               providerMsg.ToolCallID = msg.ChatToolMessage.ToolCallID
           }
   
           providerMessages = append(providerMessages, providerMsg)
       }
   
       // Build the request
       providerReq := &[ProviderName]ChatRequest{
           Model:    bifrostReq.Model,
           Messages: providerMessages,
       }
   
       // Map parameters
       if bifrostReq.Params != nil {
           params := bifrostReq.Params
   
           // Standard parameters
           if params.Temperature != nil {
               providerReq.Temperature = params.Temperature
           }
           if params.MaxTokens != nil {
               providerReq.MaxTokens = params.MaxTokens
           }
           if params.TopP != nil {
               providerReq.TopP = params.TopP
           }
           if params.FrequencyPenalty != nil {
               providerReq.FrequencyPenalty = params.FrequencyPenalty
           }
           if params.PresencePenalty != nil {
               providerReq.PresencePenalty = params.PresencePenalty
           }
           if params.Stop != nil {
               providerReq.Stop = params.Stop
           }
           if params.Seed != nil {
               providerReq.Seed = params.Seed
           }
   
           // Tool/Function calling - omitted for brevity; see complete provider examples
       }
   
       return providerReq
   }
   

Key conversion patterns to implement:

// Request Converter - Maps Bifrost standard to provider format
func To[ProviderName][Feature]Request(bifrostReq) *[ProviderName]Request {
    // 1. Nil check
    // 2. Convert messages/input
    // 3. Map standard parameters (temp, max_tokens, etc.)
    // 4. Map tools/functions if supported
    // 5. Map ExtraParams to provider-specific fields
    return providerReq
}

// Response Converter - Maps provider format back to Bifrost
func ToBifrost[Feature]Response(providerResp) (*schemas.BifrostResponse, *schemas.BifrostError) {
    // 1. Nil check with error return
    // 2. Convert choices/results
    // 3. Convert messages/content
    // 4. Convert tool calls if present
    // 5. Convert usage/metadata
    return bifrostResp, nil
}

Converter Checklist for Each Feature File:

• ✅ Request converter: To[ProviderName][Feature]Request
• ✅ Response converter: ToBifrost[Feature]Response
• ✅ Nil checks at start of every function
• ✅ Pre-allocate slices with capacity
• ✅ Handle all optional fields with nil checks
• ✅ Map ExtraParams to provider-specific fields
• ✅ Return errors for response converters
• ✅ Document complex transformations

See actual implementation examples in core/providers/huggingface/, core/providers/anthropic/, or other existing providers for complete patterns.

---

Phase 6: Implement Provider (provider_name.go)

7. Create [provider_name].go - Wire Everything Together:

   See detailed structure in “File Conventions & Responsibilities” section above.

   Implementation Checklist:

   • ✅ Package comment at top
   • ✅ All imports organized (stdlib, external, internal)
   • ✅ Provider struct with correct field order
   • ✅ Response pools (if using sync.Pool)
   • ✅ Constructor with proper initialization
   • ✅ GetProviderKey() method
   • ✅ All interface methods implemented
   • ✅ Each method follows the strict order: convert → execute → handle errors → convert back

---

Phase 7: Add Tests

8. Create [provider_name]_test.go:

   See “Adding Automated Tests” section below for complete details.

---

For OpenAI-compatible Providers

For OpenAI-compatible providers, follow the simpler structure shown in the “OpenAI-compatible Providers” section above.

Implementation Checklist:

• ✅ Create [provider_name].go only
• ✅ Import github.com/maximhq/bifrost/core/providers/openai
• ✅ Implement constructor returning (*Provider, error)
• ✅ Set default BaseURL specific to provider
• ✅ Delegate all methods to openai.HandleOpenAI* functions
• ✅ Return errors for unsupported features
• ✅ Create [provider_name]_test.go

Adding to UI

Once your provider is implemented and tested, you need to integrate it into the Bifrost UI and CI/CD pipelines.

---

Step 1: Update UI Constants

a. Add Model Placeholder (ui/lib/constants/config.ts)

Add a model placeholder example for your provider to help users understand the expected model format:

export const ModelPlaceholders = {
    openai: "e.g. gpt-4, gpt-3.5-turbo",
    anthropic: "e.g. claude-3-opus, claude-3-sonnet",
    // ... other providers
    [providername]: "e.g. model-1, model-2",  // Add your provider here
};

Example:

huggingface: "e.g. google/gemma-2-2b-it, nebius/Qwen/Qwen3-Embedding-8B",

b. Set Key Requirement (ui/lib/constants/config.ts)

Specify whether your provider requires an API key:

export const isKeyRequiredByProvider: Record<ProviderName, boolean> = {
    openai: true,
    anthropic: true,
    // ... other providers
    [providername]: true,  // Set to true if API key is required, false otherwise
};

Example:

huggingface: true,  // HuggingFace requires API key
ollama: false,      // Ollama doesn't require API key (local)

---

Step 2: Add Provider Icon (ui/lib/constants/icons.tsx)

Create an SVG icon for your provider. You can use the provider’s official brand icon or a placeholder.

export const ProviderIcons = {
    // ... existing providers
    
    [providername]: ({ size = "md", className = "" }: IconProps) => {
        const resolvedSize = resolveSize(size);
        
        return (
            <svg height="1em" viewBox="0 0 24 24" width="1em" xmlns="http://www.w3.org/2000/svg">
                <title>ProviderName</title>
                {/* Add your SVG path here */}
                <path d="..." fill="#HexColor"></path>
            </svg>
        );
    },
} as const;

Tips:

• Get the official icon from the provider’s brand assets or press kit
• Ensure the SVG is properly formatted and viewBox is set to “0 0 24 24”
• Use the provider’s brand color for the fill attribute
• Keep the icon simple and recognizable at small sizes

---

Step 3: Register Provider Name (ui/lib/constants/logs.ts)

a. Add to Known Providers List

export const KnownProvidersNames = [
    "anthropic",
    "azure",
    "bedrock",
    // ... other providers
    "[providername]",  // Add your provider name (lowercase)
] as const;

b. Add Provider Label

export const ProviderLabels: Record<ProviderName, string> = {
    anthropic: "Anthropic",
    azure: "Azure",
    bedrock: "AWS Bedrock",
    // ... other providers
    [providername]: "ProviderName",  // Add display name (proper capitalization)
} as const;

Example:

huggingface: "HuggingFace",
cerebras: "Cerebras",

---

Step 4: Update OpenAPI Specification (docs/openapi/openapi.json)

Add your provider to the API documentation’s provider enum:

{
    "type": "string",
    "enum": [
        "openai",
        "anthropic",
        "azure",
        "bedrock",
        // ... other providers
        "[providername]"
    ],
    "description": "AI model provider",
    "example": "openai"
}

Location: Search for the "AI model provider" description in docs/openapi/openapi.json and add your provider to the enum array.

---

Step 5: Update Configuration Schema (transports/config.schema.json)

a. Add Provider to Providers Object

{
    "providers": {
        "type": "object",
        "properties": {
            "openai": { "$ref": "#/$defs/provider" },
            "anthropic": { "$ref": "#/$defs/provider" },
            // ... other providers
            "[providername]": { "$ref": "#/$defs/provider" }
        }
    }
}

b. Add to Fallback Provider Enum

{
    "fallbacks": {
        "items": {
            "properties": {
                "provider": {
                    "type": "string",
                    "enum": [
                        "openai",
                        "anthropic",
                        // ... other providers
                        "[providername]"
                    ]
                }
            }
        }
    }
}

Location: Search for "fallbacks" in transports/config.schema.json and add your provider to both locations.

---

Step 6: Update UI README (ui/README.md)

Add your provider to the list of supported providers:

## Provider Configuration

Manage all your AI providers from a unified interface:

- **Supported Providers**: OpenAI, Azure, Anthropic, AWS Bedrock, Cohere, 
  Google Vertex AI, Mistral, Ollama, Parasail, Elevenlabs, SGLang, Cerebras, 
  Groq, Gemini, OpenRouter, ProviderName

Example:

- **Supported Providers**: OpenAI, Azure, Anthropic, AWS Bedrock, Cohere, 
  Google Vertex AI, Mistral, Ollama, Parasail, Elevenlabs, SGLang, Cerebras, 
  Groq, Gemini, OpenRouter, HuggingFace

---

Step 7: Register Provider in Core (core/bifrost.go)

a. Add Provider Import

import (
    // ... existing imports
    "github.com/maximhq/bifrost/core/providers/[providername]"
)

b. Add Case to createBaseProvider

func (bifrost *Bifrost) createBaseProvider(providerKey schemas.ModelProvider, config *schemas.ProviderConfig) (schemas.Provider, error) {
    // ... existing cases
    
    case schemas.ProviderName:
        return providername.NewProviderNameProvider(config, bifrost.logger), nil
    
    default:
        return nil, fmt.Errorf("unsupported provider: %s", targetProviderKey)
}

For OpenAI-compatible providers (returns error):

case schemas.ProviderName:
    return providername.NewProviderNameProvider(config, bifrost.logger)

For non-OpenAI-compatible providers (no error):

case schemas.ProviderName:
    return providername.NewProviderNameProvider(config, bifrost.logger), nil

---

Step 8: Add CI/CD Environment Variables

Add your provider’s API key to all GitHub Actions workflow files that run tests.

Files to Update:

1. .github/workflows/pr-tests.yml
2. .github/workflows/release-pipeline.yml (multiple jobs)

Changes Required:

Add the environment variable to the env: section:

env:
    OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
    ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
    # ... other API keys
    PROVIDER_NAME_API_KEY: ${{ secrets.PROVIDER_NAME_API_KEY }}

Example from pr-tests.yml:

env:
    OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
    ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
    AZURE_OPENAI_API_KEY: ${{ secrets.AZURE_OPENAI_API_KEY }}
    HUGGING_FACE_API_KEY: ${{ secrets.HUGGING_FACE_API_KEY }}

Locations in release-pipeline.yml:

• core-release job
• framework-release job
• plugins-release job
• bifrost-http-release job

Note: Repository maintainers need to add the actual secret value in GitHub repository settings under Settings > Secrets and variables > Actions.

---

UI Integration Checklist

Before submitting your PR, verify all UI changes:

• ✅ Model placeholder added to ui/lib/constants/config.ts
• ✅ Key requirement set in ui/lib/constants/config.ts
• ✅ Provider icon added to ui/lib/constants/icons.tsx
• ✅ Provider name added to ui/lib/constants/logs.ts (KnownProvidersNames)
• ✅ Provider label added to ui/lib/constants/logs.ts (ProviderLabels)
• ✅ Provider added to OpenAPI spec enum (docs/openapi/openapi.json)
• ✅ Provider added to config schema (transports/config.schema.json) - 2 locations
• ✅ Provider listed in UI README (ui/README.md)
• ✅ Provider import added to core/bifrost.go
• ✅ Provider case added to createBaseProvider in core/bifrost.go
• ✅ Environment variable added to .github/workflows/pr-tests.yml
• ✅ Environment variable added to .github/workflows/release-pipeline.yml (4 jobs)

---

Creating Provider Documentation

MANDATORY: Every new provider must have comprehensive documentation in the docs directory. This documentation helps users understand how the provider works, what parameters it supports, and any special considerations.

---

Documentation File Location

Create a new MDX file at: docs/providers/supported-providers/[provider_name].mdx

Example: For a provider named “example”, create: docs/providers/supported-providers/example.mdx

---

Documentation Structure

Your provider documentation should follow this structure for consistency. Reference complete examples:

• Groq: docs/providers/supported-providers/groq.mdx (OpenAI-compatible provider)
• Bedrock: docs/providers/supported-providers/bedrock.mdx (Custom API provider with multiple features)
• Cerebras: docs/providers/supported-providers/cerebras.mdx (OpenAI-compatible, simple)
• Mistral: docs/providers/supported-providers/mistral.mdx (Transcription + chat support)
• Ollama: docs/providers/supported-providers/ollama.mdx (Local-first infrastructure)

Required Sections

1. Front Matter (Frontmatter)

---
title: "[Provider Full Name]"
description: "[Brief description] - parameter mapping, [key features], and [auth method]"
icon: "[icon letter or emoji]"
---

Example:

---
title: "Groq"
description: "Groq API conversion guide - OpenAI-compatible format, parameter handling, text completion fallback, streaming, and tool support"
icon: "g"
---

2. Overview Section

Start with a brief overview explaining:

• What the provider is and its key characteristics
• How Bifrost converts requests to/from this provider’s format
• List of major transformation features

Template:

## Overview

[Provider Name] is a **[type: OpenAI-compatible/custom API/local-first]** provider offering [key features]. Bifrost converts requests to [Provider]'s expected format with [specific features]. Key characteristics:
- **[Feature 1]** - brief description
- **[Feature 2]** - brief description
- **[Feature 3]** - brief description

3. Supported Operations Table

Create a table showing which operations are supported:

### Supported Operations

| Operation | Non-Streaming | Streaming | Endpoint | Notes |
|-----------|---------------|-----------|----------|-------|
| Chat Completions | ✅ | ✅ | `/v1/chat/completions` | |
| Text Completions | ❌ | ❌ | Not supported | |
| Embeddings | ✅ | ❌ | `/v1/embeddings` | |
| List Models | ✅ | ❌ | `/v1/models` | |

4. Feature Sections (One per Supported Feature)

For each major feature (Chat Completions, Embeddings, etc.):

a. Request Parameters

## Request Parameters

### Parameter Mapping

| Parameter | Transformation | Notes |
|-----------|----------------|-------|
| `max_completion_tokens` | Direct pass-through | Minimum X tokens |
| `temperature` | Renamed to `temp` | Provider-specific name |

Include:

• OpenAI parameter name
• How it’s transformed for the provider (renamed, dropped, etc.)
• Any special notes or constraints

b. Filtered/Dropped Parameters

### Filtered Parameters

Removed for [Provider] compatibility:
- `prompt_cache_key` - Not supported
- `store` - Not supported

c. Special Features

### [Feature Name]

Document any provider-specific features like:
- Reasoning/thinking support
- Special authentication
- Unique parameters
- Format conversions

d. Message Conversion

## Message Conversion

Content types supported:
- ✅ Text content
- ✅ Images (URL and base64)
- ❌ Audio input

e. Response Conversion

## Response Conversion

Field mapping from provider format back to Bifrost standard.

5. Streaming Section (If Supported)

## Streaming

[Provider] uses **[protocol: SSE/WebSocket/custom]** streaming with:
- Request configuration: stream: true
- Event format: [description]
- End marker: [description]

6. Authentication Section

## Authentication

**[Authentication Type]:**

Authorization: [format]

[Additional details about key management, etc.]

7. Configuration Section

## Configuration

**HTTP Settings:**
- **Base URL**: `[default URL]` (default)
- **API Version**: [version info]
- **Max Connections**: 5000 per host
- **Idle Timeout**: 60 seconds

8. Caveats/Important Notes

Use collapsible accordion sections for limitations:

## Caveats

<Accordion title="[Caveat Title]">
**Severity**: [High/Medium/Low]
**Behavior**: [What happens]
**Impact**: [What breaks/changes]
**Code**: [File references if relevant]
</Accordion>

<Accordion title="[Another Caveat]">
...
</Accordion>

Common caveats to document:

• Unsupported content types (images, audio, etc.)
• Parameter limitations
• Streaming restrictions
• Special handling required
• Breaking behavioral differences from OpenAI standard

9. Warnings/Notes

Use special callouts for important information:

<Warning>
**Unsupported Operations**: [List operations], [List operations].
</Warning>

<Note>
[Provider] requires [special setup/configuration].
</Note>

<Tip>
[Helpful tip for using this provider effectively].
</Tip>

Code Examples in Documentation

Include examples in both formats where applicable:

<Tabs>
<Tab title="Gateway">

`````bash
curl -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "[provider]/[model]",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

</Tab> <Tab title=“Go SDK”>

response, err := client.ChatCompletion(ctx, &schemas.BifrostChatRequest{
    Provider: schemas.ProviderName,
    Model:    "model-name",
    Input:    messages,
})

</Tab> </Tabs>

### Documentation Formatting Standards

- **Markdown**: Use standard MDX syntax compatible with the documentation site
- **Code blocks**: Always specify language (bash, go, json, yaml)
- **Tables**: Use pipes for alignment and clarity
- **Sections**: Use H1 (#) for main provider, H2 (##) for major sections, H3 (###) for subsections
- **Lists**: Use bullets (-) for unordered, numbers (1.) for ordered
- **Emphasis**: Use **bold** for important terms, `code` for inline code

### Documentation Checklist

Before submitting your documentation:

- ✅ File created: `docs/providers/supported-providers/[provider_name].mdx`
- ✅ Front matter with title, description, and icon
- ✅ Overview section explaining the provider
- ✅ Supported Operations table (accurate for your implementation)
- ✅ Parameter mapping documented for each supported feature
- ✅ Filtered parameters listed
- ✅ Message conversion explained (content types)
- ✅ Tool/function support documented (if applicable)
- ✅ Response conversion patterns explained
- ✅ Streaming behavior documented (if supported)
- ✅ Authentication method clearly explained
- ✅ Configuration section with base URL and settings
- ✅ Caveats documented with severity ratings
- ✅ Code examples for Gateway and Go SDK (where applicable)
- ✅ All special features explained
- ✅ Links to reference existing implementations (if applicable)
- ✅ Warnings for unsupported features

---

## Adding Automated Tests

Testing is **MANDATORY** for all providers. Tests ensure your provider works correctly and continues to work as the codebase evolves.

---

### Test File Structure

Create `core/providers/[provider_name]/[provider_name]_test.go`:

**Example Test File Structure**:

```go
package providername_test

import (
    "os"
    "testing"

    "github.com/maximhq/bifrost/core/internal/llmtests"
    "github.com/maximhq/bifrost/core/schemas"
)

func TestProviderName(t *testing.T) {
    t.Parallel()
    
    // Check for API key - skip if not available
    if os.Getenv("PROVIDER_API_KEY") == "" {
        t.Skip("Skipping tests because PROVIDER_API_KEY is not set")
    }

    // Initialize test client
    client, ctx, cancel, err := llmtests.SetupTest()
    if err != nil {
        t.Fatalf("Error initializing test setup: %v", err)
    }
    defer cancel()

    // Configure test scenarios
    testConfig := llmtests.ComprehensiveTestConfig{
        Provider:  schemas.ProviderName,
        ChatModel: "model-name",
        Fallbacks: []schemas.Fallback{
            {Provider: schemas.ProviderName, Model: "fallback-model"},
        },
        Scenarios: llmtests.TestScenarios{
            SimpleChat:            true,
            CompletionStream:      true,
            ToolCalls:             true,
            TextCompletion:        false,  // Not supported
            Embedding:             false,  // Not supported
            ListModels:            true,
            // ... configure based on provider capabilities
        },
    }

    // Run all tests
    t.Run("ProviderNameTests", func(t *testing.T) {
        llmtests.RunAllComprehensiveTests(t, client, ctx, testConfig)
    })
    
    client.Shutdown()
}

---

Test Configuration Requirements

Environment Variables:

• REQUIRED: [PROVIDER_NAME]_API_KEY - API key for the provider
• Optional: PROVIDER_BASE_URL - Custom base URL for testing

Example:

export CEREBRAS_API_KEY="your-api-key-here"
export HUGGING_FACE_API_KEY="your-hf-token-here"

Package Declaration:

• Use package [provider_name]_test (note the _test suffix)
• This ensures tests don’t access unexported functions (tests external behavior)

See complete test examples in core/providers/cerebras/cerebras_test.go, core/providers/huggingface/huggingface_test.go, or other existing providers.

---

Test Scenarios Configuration

The llmtests.TestScenarios struct defines which tests to run. Set each field based on provider capabilities:

Core Test Scenarios

Scenario	Enable if…
SimpleChat	Provider supports basic chat completion
CompletionStream	Provider supports streaming chat
TextCompletion	Provider supports text completions (legacy)
TextCompletionStream	Provider supports streaming text completions
ToolCalls	Provider supports function/tool calling
ToolCallsStreaming	Provider supports streaming with tool calls
Embedding	Provider supports text embeddings
ListModels	Provider has a list models endpoint
ImageURL	Provider accepts image URLs in messages
ImageBase64	Provider accepts base64-encoded images

For a complete list of all available test scenarios and their descriptions, check the llmtests.TestScenarios struct in core/internal/llmtests/.

---

Model Configuration

ChatModel (REQUIRED if any chat scenario is enabled):

ChatModel: "llama-3.3-70b",    // Primary model for chat tests

TextModel (REQUIRED if any text completion scenario is enabled):

TextModel: "llama3.1-8b",      // Model for text completion tests

EmbeddingModel (REQUIRED if Embedding scenario is enabled):

EmbeddingModel: "text-embedding-ada-002",  // Model for embedding tests

Fallbacks (OPTIONAL but recommended):

Fallbacks: []schemas.Fallback{
    {Provider: schemas.Cerebras, Model: "llama3.1-8b"},
    {Provider: schemas.Cerebras, Model: "gpt-oss-120b"},
},

• Fallbacks are tested if primary model fails
• Tests that fallback mechanism works correctly

---

Running Tests

Run all tests for your provider:

cd core/providers/[provider_name]
go test -v

Run with API key:

PROVIDER_API_KEY="your-key" go test -v

Run specific test:

go test -v -run TestProviderName

Run with timeout (for slow providers):

go test -v -timeout 5m

Skip integration tests (if API key not set):

go test -v -short

---

Test Checklist

Before submitting your provider, ensure:

• ✅ Test file named [provider_name]_test.go
• ✅ Package is [provider_name]_test
• ✅ t.Parallel() called at start
• ✅ API key check with t.Skip() if not available
• ✅ All supported scenarios enabled in config
• ✅ All unsupported scenarios disabled (set to false)
• ✅ Appropriate models specified (ChatModel, TextModel, EmbeddingModel)
• ✅ Fallback models configured (at least 1-2)
• ✅ client.Shutdown() called at end
• ✅ Tests pass locally with valid API key
• ✅ Tests skip gracefully without API key

---

Common Test Failures and Solutions

Test hangs indefinitely:

• Solution: Add timeout: go test -v -timeout 2m
• Cause: Provider not responding or network issue

“API key not set” skip message:

• Solution: Export the required environment variable
• Not a failure: Tests correctly skip when credentials unavailable

“Unsupported feature” errors:

• Solution: Set the scenario to false in TestScenarios
• Cause: Test trying to run unsupported feature

“Model not found” errors:

• Solution: Update ChatModel/TextModel to valid model for provider
• Cause: Model name incorrect or not available

Streaming tests fail but non-streaming pass:

• Solution: Check streaming implementation in provider
• Cause: SSE parsing error or incorrect stream handling

Tool calling tests fail:

• Solution: Verify tool/function conversion in chat.go
• Cause: Tool format doesn’t match provider’s expected structure

---

CI/CD Integration

After your tests pass locally, ensure they’ll run in the CI/CD pipeline:

GitHub Actions Setup

Required Secret: Your provider’s API key must be added to GitHub repository secrets by a maintainer:

• Secret name: PROVIDER_NAME_API_KEY (uppercase, underscores)
• Example: HUGGING_FACE_API_KEY, CEREBRAS_API_KEY

Workflow Files: The API key environment variable should already be added if you followed Step 8 in “Adding to UI”. Verify it’s present in:

• .github/workflows/pr-tests.yml
• .github/workflows/release-pipeline.yml (4 jobs)

Test Execution: Tests will automatically run on:

• Pull requests (PR tests workflow)
• Release builds (release pipeline workflow)
• Manual workflow triggers

Skipping Tests: If the API key secret is not set, your tests will be skipped (not fail) thanks to the t.Skip() check in your test file.

---

Final Pre-Submission Checklist

Before creating a pull request, verify everything is complete:

Provider Implementation:

• ✅ Provider code follows file structure conventions
• ✅ All supported features implemented correctly
• ✅ Error handling properly converts to schemas.BifrostError
• ✅ OpenAI handlers used if provider is compatible
• ✅ Code is well-commented and documented

Tests:

• ✅ Test file created: [provider_name]_test.go
• ✅ All supported scenarios enabled
• ✅ All unsupported scenarios disabled
• ✅ Tests pass locally with valid API key
• ✅ Tests skip gracefully without API key
• ✅ Appropriate models configured

Schema & Core:

• ✅ Provider added to core/schemas/bifrost.go (ModelProvider type + arrays)
• ✅ Provider registered in core/bifrost.go (import + case)

UI Integration:

• ✅ All 7 UI files updated (config.ts, icons.tsx, logs.ts, etc.)
• ✅ Provider icon looks good and is recognizable
• ✅ Model placeholders are helpful examples

CI/CD:

• ✅ Environment variables added to workflow files
• ✅ API key secret name follows convention

Documentation:

• ✅ Provider-specific parameters documented (if any)
• ✅ Example usage added (optional but helpful)
• ✅ Any special setup instructions noted

---