Skip to main content
Since LiteLLM already provides multi-provider abstraction, Bifrost adds enterprise features like governance, semantic caching, MCP tools, observability, etc, on top of your existing setup. Endpoint: /litellm
Provider Compatibility: This integration only works for AI providers that both LiteLLM and Bifrost support. If you’re using a provider specific to LiteLLM that Bifrost doesn’t support (or vice versa), those requests will fail.

​
Setup

  • Python
from litellm import completion

# Configure client to use Bifrost
response = completion(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello!"}],
    base_url="http://localhost:8080/litellm"  # Point to Bifrost
)

print(response.choices[0].message.content)

​
Provider/Model Usage Examples

Your existing LiteLLM provider switching works unchanged through Bifrost:
  • Python
from litellm import completion

# All your existing LiteLLM patterns work the same
base_url = "http://localhost:8080/litellm"

# OpenAI models
openai_response = completion(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello GPT!"}],
    base_url=base_url
)

# Anthropic models  
anthropic_response = completion(
    model="claude-3-sonnet-20240229",
    messages=[{"role": "user", "content": "Hello Claude!"}],
    base_url=base_url
)

# Google models
google_response = completion(
    model="gemini/gemini-1.5-flash",
    messages=[{"role": "user", "content": "Hello Gemini!"}],
    base_url=base_url
)

# Azure OpenAI models
azure_response = completion(
    model="azure/gpt-4o",
    messages=[{"role": "user", "content": "Hello Azure!"}],
    base_url=base_url
)

​
Adding Custom Headers

Add Bifrost-specific headers for governance and tracking:
  • Python
from litellm import completion

# Add custom headers for Bifrost features
response = completion(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello!"}],
    base_url="http://localhost:8080/litellm",
    extra_headers={
        "x-bf-vk": "your-virtual-key",          # Virtual key for governance
    }
)

print(response.choices[0].message.content)

​
Using Direct Keys

Pass API keys directly to bypass Bifrost’s key management. You can pass any provider’s API key since Bifrost only looks for Authorization or x-api-key headers. This requires the Allow Direct API keys option to be enabled in Bifrost configuration.
Learn more: See Quickstart Configuration for enabling direct API key usage.
  • Python
from litellm import completion

# Using OpenAI key directly
openai_response = completion(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello GPT!"}],
    base_url="http://localhost:8080/litellm",
    extra_headers={
        "Authorization": "Bearer sk-your-openai-key"
    }
)

# Using Anthropic key for Claude models
anthropic_response = completion(
    model="claude-3-sonnet-20240229", 
    messages=[{"role": "user", "content": "Hello Claude!"}],
    base_url="http://localhost:8080/litellm",
    extra_headers={
        "x-api-key": "sk-ant-your-anthropic-key"
    }
)

# Using Azure OpenAI with direct Azure key
import os

deployment = os.getenv("AZURE_OPENAI_DEPLOYMENT", "my-azure-deployment")
model = f"azure/{deployment}"

azure_response = completion(
    model=model,
    messages=[{"role": "user", "content": "Hello from LiteLLM (Azure demo)!"}],
    base_url="http://localhost:8080/litellm",
    api_key=os.getenv("AZURE_API_KEY", "your-azure-api-key"),
    deployment_id=os.getenv("AZURE_OPENAI_DEPLOYMENT", "gpt-4o-aug"),
    max_tokens=100,
    extra_headers={
        "x-bf-azure-endpoint": "https://your-resource.openai.azure.com",
    }
)

​
Supported Features

The LiteLLM integration supports all features that are available in both the LiteLLM SDK and Bifrost core functionality. Your existing LiteLLM code works seamlessly with Bifrost’s enterprise features. 😄

​
Next Steps