LiteLLM SDK - Vikat

Since LiteLLM already provides multi-provider abstraction, Vikat adds enterprise features like governance, semantic caching, MCP tools, observability, etc, on top of your existing setup. Endpoint: /litellm

Provider Compatibility: This integration only works for AI providers that both LiteLLM and Vikat support. If you’re using a provider specific to LiteLLM that Vikat doesn’t support (or vice versa), those requests will fail.

Setup

Python

from litellm import completion

# Configure client to use Vikat
response = completion(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello!"}],
    base_url="http://localhost:8080/litellm"  # Point to Vikat
)

print(response.choices[0].message.content)

Provider/Model Usage Examples

Your existing LiteLLM provider switching works unchanged through Vikat:

Python

from litellm import completion

# All your existing LiteLLM patterns work the same
base_url = "http://localhost:8080/litellm"

# OpenAI models
openai_response = completion(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello GPT!"}],
    base_url=base_url
)

# Anthropic models  
anthropic_response = completion(
    model="claude-3-sonnet-20240229",
    messages=[{"role": "user", "content": "Hello Claude!"}],
    base_url=base_url
)

# Google models
google_response = completion(
    model="gemini/gemini-1.5-flash",
    messages=[{"role": "user", "content": "Hello Gemini!"}],
    base_url=base_url
)

# Azure models
azure_response = completion(
    model="azure/gpt-4o",
    messages=[{"role": "user", "content": "Hello Azure!"}],
    base_url=base_url
)

Adding Custom Headers

Add Vikat-specific headers for governance and tracking:

Python

from litellm import completion

# Add custom headers for Vikat features
response = completion(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello!"}],
    base_url="http://localhost:8080/litellm",
    extra_headers={
        "x-vikat-vk": "your-virtual-key",          # Virtual key for governance
    }
)

print(response.choices[0].message.content)

Supported Features

The LiteLLM integration supports all features that are available in both the LiteLLM SDK and Vikat core functionality. Your existing LiteLLM code works seamlessly with Vikat’s enterprise features. 😄

Next Steps

Governance Features - Virtual keys and team management
Semantic Caching - Intelligent response caching
Configuration - Provider setup and API key management