> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/getzep/graphiti/llms.txt
> Use this file to discover all available pages before exploring further.

# LLM Provider Configuration

> Configure OpenAI, Anthropic, Google Gemini, Groq, or Azure OpenAI for entity extraction and summarization

Graphiti uses large language models (LLMs) to extract entities, relationships, and generate summaries from your content. Configure your preferred LLM provider to power these operations.

## Supported Providers

<CardGroup cols={2}>
  <Card title="OpenAI" icon="openai">
    GPT-4o, GPT-4o-mini, GPT-5, and more
  </Card>

  <Card title="Azure OpenAI" icon="microsoft">
    Enterprise OpenAI models on Azure
  </Card>

  <Card title="Anthropic" icon="brain">
    Claude 3.5 Sonnet, Claude 3 Opus, and more
  </Card>

  <Card title="Google Gemini" icon="google">
    Gemini Pro and Gemini Flash
  </Card>

  <Card title="Groq" icon="bolt">
    Fast inference with Llama, Mixtral, and more
  </Card>
</CardGroup>

## Default Provider (OpenAI)

By default, Graphiti uses OpenAI's GPT-4o-mini:

```python theme={null}
from graphiti_core import Graphiti
import os

# Set your API key
os.environ["OPENAI_API_KEY"] = "sk-..."

# Uses OpenAI by default
graphiti = Graphiti(
    uri="bolt://localhost:7687",
    user="neo4j",
    password="password"
)
```

## OpenAI Configuration

### Basic Setup

```python theme={null}
from graphiti_core import Graphiti
from graphiti_core.llm_client import OpenAIClient, LLMConfig

# Configure OpenAI client
llm_config = LLMConfig(
    api_key="sk-...",
    model="gpt-4o",
    small_model="gpt-4o-mini",
    temperature=0.7,
    max_tokens=4096
)

llm_client = OpenAIClient(config=llm_config)

graphiti = Graphiti(
    uri="bolt://localhost:7687",
    user="neo4j",
    password="password",
    llm_client=llm_client
)
```

### Configuration Options

| Parameter     | Type    | Default         | Description                  |
| ------------- | ------- | --------------- | ---------------------------- |
| `api_key`     | `str`   | From env        | OpenAI API key               |
| `model`       | `str`   | `"gpt-4o-mini"` | Primary model for extraction |
| `small_model` | `str`   | `"gpt-4o-nano"` | Model for smaller tasks      |
| `temperature` | `float` | `0.0`           | Sampling temperature (0-2)   |
| `max_tokens`  | `int`   | `4096`          | Maximum tokens per request   |
| `base_url`    | `str`   | `None`          | Custom API endpoint          |

### Environment Variables

```bash theme={null}
OPENAI_API_KEY=sk-...
```

### Recommended Models

* `gpt-4o` - Best quality for complex extraction
* `gpt-4o-mini` - Balanced performance and cost
* `gpt-5-mini` - Fast extraction with good quality
* `o1-mini` - Reasoning model for complex relationships

## Azure OpenAI

Use OpenAI models deployed on Azure:

```python theme={null}
from graphiti_core import Graphiti
from graphiti_core.llm_client.azure_openai_client import AzureOpenAILLMClient
from graphiti_core.llm_client.config import LLMConfig
from openai import AsyncOpenAI

# Create Azure OpenAI client
azure_client = AsyncOpenAI(
    base_url="https://your-resource.openai.azure.com/openai/v1/",
    api_key="your-azure-api-key"
)

# Configure LLM client
llm_client = AzureOpenAILLMClient(
    azure_client=azure_client,
    config=LLMConfig(
        model="gpt-4.1",  # Your Azure deployment name
        small_model="gpt-4.1"
    )
)

graphiti = Graphiti(
    uri="bolt://localhost:7687",
    user="neo4j",
    password="password",
    llm_client=llm_client
)
```

### Environment Variables

```bash theme={null}
AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com
AZURE_OPENAI_API_KEY=your-key
AZURE_OPENAI_DEPLOYMENT=gpt-4.1
```

## Anthropic (Claude)

Use Claude models for extraction:

```python theme={null}
from graphiti_core import Graphiti
from graphiti_core.llm_client import AnthropicClient, LLMConfig

llm_config = LLMConfig(
    api_key="sk-ant-...",
    model="claude-3-5-sonnet-20241022",
    small_model="claude-3-5-haiku-20241022",
    temperature=0.0,
    max_tokens=4096
)

llm_client = AnthropicClient(config=llm_config)

graphiti = Graphiti(
    uri="bolt://localhost:7687",
    user="neo4j",
    password="password",
    llm_client=llm_client
)
```

### Environment Variables

```bash theme={null}
ANTHROPIC_API_KEY=sk-ant-...
```

### Recommended Models

* `claude-3-5-sonnet-20241022` - Best quality and reasoning
* `claude-3-5-haiku-20241022` - Fast and cost-effective
* `claude-3-opus-20240229` - Maximum capability

## Google Gemini

Use Google's Gemini models:

```python theme={null}
from graphiti_core import Graphiti
from graphiti_core.llm_client.gemini_client import GeminiClient
from graphiti_core.llm_client.config import LLMConfig

llm_config = LLMConfig(
    api_key="your-google-api-key",
    model="gemini-1.5-pro",
    small_model="gemini-1.5-flash",
    temperature=0.0,
    max_tokens=4096
)

llm_client = GeminiClient(config=llm_config)

graphiti = Graphiti(
    uri="bolt://localhost:7687",
    user="neo4j",
    password="password",
    llm_client=llm_client
)
```

### Environment Variables

```bash theme={null}
GOOGLE_API_KEY=your-key
```

### Recommended Models

* `gemini-1.5-pro` - Best quality
* `gemini-1.5-flash` - Fast inference
* `gemini-2.0-flash` - Latest fast model

## Groq

Use Groq for ultra-fast inference:

```python theme={null}
from graphiti_core import Graphiti
from graphiti_core.llm_client.groq_client import GroqClient
from graphiti_core.llm_client.config import LLMConfig

llm_config = LLMConfig(
    api_key="gsk_...",
    model="llama-3.3-70b-versatile",
    small_model="llama-3.1-8b-instant",
    temperature=0.0,
    max_tokens=4096
)

llm_client = GroqClient(config=llm_config)

graphiti = Graphiti(
    uri="bolt://localhost:7687",
    user="neo4j",
    password="password",
    llm_client=llm_client
)
```

### Environment Variables

```bash theme={null}
GROQ_API_KEY=gsk_...
```

### Recommended Models

* `llama-3.3-70b-versatile` - Best Llama model
* `llama-3.1-8b-instant` - Fast inference
* `mixtral-8x7b-32768` - Good for long contexts

## Custom Base URLs

Use custom endpoints for OpenAI-compatible APIs:

```python theme={null}
from graphiti_core.llm_client import OpenAIClient, LLMConfig

llm_config = LLMConfig(
    api_key="your-key",
    model="custom-model-name",
    base_url="https://api.your-provider.com/v1"
)

llm_client = OpenAIClient(config=llm_config)
```

This works with:

* OpenRouter
* Together AI
* Local LLM servers (Ollama, vLLM, etc.)
* Any OpenAI-compatible API

## Token Tracking

Graphiti tracks token usage across all LLM calls:

```python theme={null}
# Add some episodes
await graphiti.add_episode(...)
await graphiti.add_episode(...)

# Get token usage summary
graphiti.token_tracker.print_summary(sort_by='prompt_name')

# Or access programmatically
usage = graphiti.token_tracker.get_total_usage()
print(f"Total tokens: {usage['total_tokens']}")
print(f"Total cost: ${usage['total_cost']:.4f}")

# Get usage by prompt type
by_prompt = graphiti.token_tracker.get_usage()
for prompt_name, stats in by_prompt.items():
    print(f"{prompt_name}: {stats['total_tokens']} tokens")

# Reset tracking
graphiti.token_tracker.reset()
```

## Model Selection Strategy

Graphiti uses two model types:

<Tabs>
  <Tab title="Primary Model">
    Used for:

    * Entity extraction
    * Relationship extraction
    * Complex reasoning

    Recommended:

    * OpenAI: `gpt-4o`
    * Anthropic: `claude-3-5-sonnet-20241022`
    * Gemini: `gemini-1.5-pro`
  </Tab>

  <Tab title="Small Model">
    Used for:

    * Simple classifications
    * Quick summaries
    * Lightweight tasks

    Recommended:

    * OpenAI: `gpt-4o-mini`
    * Anthropic: `claude-3-5-haiku-20241022`
    * Gemini: `gemini-1.5-flash`
  </Tab>
</Tabs>

## Cost Optimization

<CardGroup cols={2}>
  <Card title="Use Small Models" icon="coins">
    Set `small_model` to a cost-effective option like `gpt-4o-mini` or `claude-3-5-haiku-20241022`
  </Card>

  <Card title="Batch Episodes" icon="layer-group">
    Use `add_episode_bulk()` to process multiple episodes efficiently
  </Card>

  <Card title="Lower Temperature" icon="temperature-low">
    Use `temperature=0.0` for deterministic, focused outputs
  </Card>

  <Card title="Track Usage" icon="chart-line">
    Monitor token usage with `token_tracker` to identify optimization opportunities
  </Card>
</CardGroup>

## Reasoning Models

For OpenAI's reasoning models (o1, o3 series), configure reasoning effort:

```python theme={null}
from graphiti_core.llm_client import OpenAIClient, LLMConfig

llm_client = OpenAIClient(
    config=LLMConfig(
        model="o1-mini",
        small_model="gpt-4o-mini"
    ),
    reasoning="medium",  # low, medium, high
    verbosity="concise"  # concise, standard, detailed
)
```

<Note>
  Reasoning models don't support the `temperature` parameter. It's automatically set to `None`.
</Note>

## Caching

Enable LLM response caching to reduce costs and latency:

```python theme={null}
from graphiti_core.llm_client import OpenAIClient, LLMConfig

llm_client = OpenAIClient(
    config=LLMConfig(model="gpt-4o-mini"),
    cache=True  # Enable caching
)
```

<Warning>
  Caching stores responses in memory. Use with caution in production environments.
</Warning>

## Error Handling

```python theme={null}
try:
    result = await graphiti.add_episode(
        name="Test",
        episode_body="Content",
        source=EpisodeType.text,
        source_description="Test",
        reference_time=datetime.now(timezone.utc)
    )
except Exception as e:
    print(f"LLM error: {e}")
    # Handle rate limits, API errors, etc.
```

## Next Steps

<CardGroup cols={2}>
  <Card title="Embeddings" icon="vector-square" href="/guides/embeddings">
    Configure embedding providers for semantic search
  </Card>

  <Card title="Graph Drivers" icon="database" href="/guides/graph-drivers">
    Choose and configure your graph database
  </Card>

  <Card title="Adding Episodes" icon="plus" href="/guides/adding-episodes">
    Start adding content to your knowledge graph
  </Card>
</CardGroup>