LiteLLM SDK

Use Bifrost as a drop-in proxy for LiteLLM applications with zero code changes.

Since LiteLLM already provides multi-provider abstraction, Bifrost adds enterprise features like governance, semantic caching, MCP tools, observability, etc, on top of your existing setup.

Endpoint: /litellm

Provider Compatibility: This integration only works for AI providers that both LiteLLM and Bifrost support. If you’re using a provider specific to LiteLLM that Bifrost doesn’t support (or vice versa), those requests will fail.

---

Setup

Python

from litellm import completion

# Configure client to use Bifrost
response = completion(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello!"}],
    base_url="http://localhost:8080/litellm"  # Point to Bifrost
)

print(response.choices[0].message.content)

---

Provider/Model Usage Examples

Your existing LiteLLM provider switching works unchanged through Bifrost:

Python

from litellm import completion

# All your existing LiteLLM patterns work the same
base_url = "http://localhost:8080/litellm"

# OpenAI models
openai_response = completion(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello GPT!"}],
    base_url=base_url
)

# Anthropic models  
anthropic_response = completion(
    model="claude-3-sonnet-20240229",
    messages=[{"role": "user", "content": "Hello Claude!"}],
    base_url=base_url
)

# Google models
google_response = completion(
    model="gemini/gemini-1.5-flash",
    messages=[{"role": "user", "content": "Hello Gemini!"}],
    base_url=base_url
)

# Azure models
azure_response = completion(
    model="azure/gpt-4o",
    messages=[{"role": "user", "content": "Hello Azure!"}],
    base_url=base_url
)

---

Adding Custom Headers

Add Bifrost-specific headers for governance and tracking:

Python

from litellm import completion

# Add custom headers for Bifrost features
response = completion(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello!"}],
    base_url="http://localhost:8080/litellm",
    extra_headers={
        "x-bf-vk": "your-virtual-key",          # Virtual key for governance
    }
)

print(response.choices[0].message.content)

---

Supported Features

The LiteLLM integration supports all features that are available in both the LiteLLM SDK and Bifrost core functionality. Your existing LiteLLM code works seamlessly with Bifrost’s enterprise features. 😄

---

Next Steps

• Governance Features - Virtual keys and team management
• Semantic Caching - Intelligent response caching
• Configuration - Provider setup and API key management