Skip to main content

OpenAI Inferences

Make inferences with Clarifai using an OpenAI-compatible format

Clarifai provides an OpenAI-compatible API endpoint, which allows you to leverage your existing OpenAI API code and workflows to make inferences with Clarifai models, including those that integrate or wrap OpenAI models.

The built-in compatibility layer converts your OpenAI calls directly into Clarifai API requests, letting you harness Clarifai's diverse models as custom tools in your OpenAI projects.

This simplifies the integration process, as you don't need to rewrite your code specifically for Clarifai's native API structure if you're already familiar with OpenAI's.

Note Usage-based billing is handled directly through Clarifai — not through OpenAI or any other external tool. Also, while most OpenAI parameters are supported, certain advanced features may be unavailable depending on the specific model or endpoint.

OpenAI​

Prerequisites​

Install Clarifai Package​

Install the latest version of the Clarifai Python SDK package:

 pip install --upgrade clarifai

Get a PAT Key​

You need a PAT (Personal Access Token) key to authenticate your connection to the Clarifai platform. You can generate the PAT key in your personal settings page by navigating to the Security section.

Install Openai Package​

Install the openai package:

 pip install openai

Example​

Here is an example that uses the OpenAI Python client library to interact with a Clarifai model via Clarifai's OpenAI-compatible API endpoint.

from openai import OpenAI

# Initialize the OpenAI client, pointing to Clarifai's API
client = OpenAI(
    api_key="YOUR_CLARIFAI_PAT_KEY_HERE",  
    base_url="https://api.clarifai.com/v2/ext/openai/v1"  # Clarifai's OpenAI-compatible API endpoint
)

# Make a chat completion request to a Clarifai-hosted model
response = client.chat.completions.create(
    model="anthropic/completion/models/claude-sonnet-4",  # Clarifai model name
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Who are you?"}
    ],
    # You can also add other OpenAI-compatible parameters like temperature, max_tokens, etc.
    max_completion_tokens=100,  # Limits the response length
    temperature=0.7,  # Controls randomness of the output
    stream=True  # Enables streaming the response token by token
)

print("Assistant's Response:")
for chunk in response:
    # Safely check if choices, delta, and content exist before accessing
    if chunk.choices and \
       chunk.choices[0].delta and \
       chunk.choices[0].delta.content is not None:
        print(chunk.choices[0].delta.content, end='')
print("\n")
Example Output
Assistant's Response:
I'm Claude, an AI assistant created by Anthropic. I'm here to help with a wide variety of tasks like answering questions, helping with analysis and research, creative projects, math and coding, and having conversations. Is there something specific I can help you with today?

Tool Calling​

Tool calling (formerly known as function calling) enables large language models (LLMs) to autonomously decide when and how to invoke external tools — such as APIs or custom functions — based on user input.

With Clarifai’s support for OpenAI-compatible APIs, you can seamlessly integrate tool-calling capabilities using your existing OpenAI workflows, while leveraging Clarifai-hosted or custom models.

Here is an example code that sets up a basic tool-calling interaction. It simulates a weather API and shows how the LLM would "call" that tool when asked about the weather.

from openai import OpenAI

# Initialize the OpenAI-compatible client for Clarifai
client = OpenAI(
    api_key="YOUR_CLARIFAI_PAT_KEY_HERE",
    base_url="https://api.clarifai.com/v2/ext/openai/v1"   
)

# Define the external tools (functions) that the LLM can call.
# In this example, it's a 'get_weather' function.
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Returns the current temperature for a given location.",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "City and country, e.g., 'Bogotá, Colombia'"
                    }
                },
                "required": ["location"],
                "additionalProperties": False # Ensures no extra parameters are passed
            }
        }
    }
]

# Create a chat completion request with tool-calling enabled
response = client.chat.completions.create(
    model="anthropic/completion/models/claude-sonnet-4",  # Clarifai-compatible OpenAI model name
    messages=[
        {"role": "user", "content": "What is the weather like in New York today?"}
    ],
    tools=tools,
    tool_choice='auto' # Let the LLM decide if it needs to use a tool
)

# Print the tool call proposed by the model, if any
tool_calls = response.choices[0].message.tool_calls
print("Tool calls:", tool_calls)
Tool Calling Implementation Example
import json
from openai import OpenAI

# Initialize the OpenAI client, pointing to Clarifai's OpenAI-compatible API endpoint
client = OpenAI(
    base_url="https://api.clarifai.com/v2/ext/openai/v1",
    api_key="c02f72c90a884d2aa25be93c75a0e6d0",
)

# Define the external tools (functions) that the LLM can call.
# In this example, it's a 'get_weather' function.
tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current temperature for a given location.",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "City and country, e.g., 'Bogotá, Colombia'"
                }
            },
            "required": ["location"],
            "additionalProperties": False # Ensures no extra parameters are passed
        },
        "strict": True # Enforces strict adherence to parameter schema
    }
}]

## Simulate Tool Execution (for demonstration)

# This function simulates calling an external weather API.
# In a real application, this would make an actual API request.
def get_weather(location: str):
    """Simulates fetching weather for a given location."""
    # Placeholder data for demonstration
    if "New York" in location:
        return {"location": "New York", "temperature": "20°C", "conditions": "Partly cloudy"}
    elif "London" in location:
        return {"location": "London", "temperature": "15°C", "conditions": "Rainy"}
    else:
        return {"location": location, "temperature": "N/A", "conditions": "Unknown"}

## LLM Call with Tooling

# First API call: The LLM decides if a tool needs to be called.
print("--- Initial LLM Call (Tool Recommendation) ---")
first_response = client.chat.completions.create(
    model="anthropic/completion/models/claude-sonnet-4", # Ensure this model supports tool calling on Clarifai's platform
    messages=[
        {"role": "user", "content": "What is the weather like in New York today?"}
    ],
    tools=tools, # Provide the list of available tools
    tool_choice="auto", # Let the LLM decide if it needs to use a tool
)


## Process LLM's Response and Execute Tool (if recommended)

# Check if the LLM decided to call a tool
if first_response.choices[0].message.tool_calls:
    tool_calls = first_response.choices[0].message.tool_calls
    print(f"\nLLM recommended tool calls: {tool_calls}")

    # Execute each recommended tool call
    available_functions = {
        "get_weather": get_weather, # Map function name to actual Python function
    }

    messages = [
        {"role": "user", "content": "What is the weather like in New York today?"}
    ]
    messages.append(first_response.choices[0].message) # Add LLM's tool call suggestion to messages

    for tool_call in tool_calls:
        function_name = tool_call.function.name
        function_to_call = available_functions[function_name]
        function_args = json.loads(tool_call.function.arguments)

        # Call the actual Python function
        function_response = function_to_call(**function_args)
        print(f"\nExecuting tool: {function_name}({function_args}) -> {function_response}")

        # Add the tool's output to the conversation for the LLM to process
        messages.append(
            {
                "tool_call_id": tool_call.id,
                "role": "tool",
                "name": function_name,
                "content": json.dumps(function_response),
            }
        )

    # ---
    ## Second LLM Call (Summarize Tool Output)
    

    # Now, send the tool's output back to the LLM to get a natural language response
    print("\n--- Second LLM Call (Summarizing Tool Output) ---")
    second_response = client.chat.completions.create(
        model="anthropic/completion/models/claude-sonnet-4",
        messages=messages, # Continue the conversation with tool output
    )

    print("\nFinal Assistant's Response:")
    print(second_response.choices[0].message.content)

else:
    print("\nLLM did not recommend any tool calls.")
    print("Assistant's direct response:")
    print(first_response.choices[0].message.content)
--- Initial LLM Call (Tool Recommendation) ---

LLM recommended tool calls: [ChatCompletionMessageToolCall(id='toolu_01Mhqb1c7ne4GPKWY9eZtgxd', function=Function(arguments='{"location": "New York, United States"}', name='get_weather'), type='function')]

Executing tool: get_weather({'location': 'New York, United States'}) -> {'location': 'New York', 'temperature': '20°C', 'conditions': 'Partly cloudy'}

--- Second LLM Call (Summarizing Tool Output) ---

Final Assistant's Response:
The weather in New York today is:
- **Temperature:** 20°C (68°F)
- **Conditions:** Partly cloudy

It's a pleasant day with mild temperatures and partly cloudy skies!

Vercel AI SDK​

The Vercel AI SDK provides a convenient way to interact with Clarifai's OpenAI-compatible API. You can leverage the OpenAI provider to interact with Clarifai models.

Example​

import { createOpenAI } from "@ai-sdk/openai";
import { generateText } from "ai";

const openai = createOpenAI({
  baseURL: "https://api.clarifai.com/v2/ext/openai/v1",
  apiKey: process.env.CLARIFAI_PAT,
});

const model = openai(
  "https://clarifai.com/anthropic/completion/models/claude-sonnet-4",
);

const { text } = await generateText({
  model,
  messages: [
    { role: "system", content: "You are a helpful assistant." },
    { role: "user", content: "What is photosynthesis?" },
  ],
});

console.log(text);

LiteLLM​

You can use the LiteLLM Python SDK to directly route inference requests to their Clarifai-hosted models. This provides a lightweight, OpenAI-compatible interface for interacting with Clarifai's powerful LLMs using a single, unified API.

To use Clarifai models via LiteLLM, you'll need to:

  • Install the Clarifai package and get a PAT key as mentioned earlier.
  • Install LiteLLM by running pip install litellm.
  • Specify Clarifai models by using the model path prefixed with openai/ followed by the Clarifai model URL (e.g., openai/https://clarifai.com/...).

Example​

import litellm

# Call litellm.completion or litellm.chat_completion to send requests
for chunk in litellm.completion(
    model="openai/https://clarifai.com/openai/chat-completion/models/o4-mini",
    api_key="CLARIFAI_PAT",
    api_base="https://api.clarifai.com/v2/ext/openai/v1",
    # Message formatting is consistent with OpenAI's schema ({"role": ..., "content": ...}).
    messages=[
        {"role": "user", "content": "Tell me a fun fact about space."}
    ],
    stream=True, # Supports streaming responses
):
    print(chunk.choices[0].delta)

Tool Calling​

Clarifai models accessed via LiteLLM also support tool calling.

import litellm

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current temperature for a given location.",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "City and country e.g. Tokyo, Japan"
                }
            },
            "required": ["location"],
            "additionalProperties": False
        },
    }
}]

response = litellm.completion(
    model="openai/https://clarifai.com/openai/chat-completion/models/o4-mini",
    api_key="CLARIFAI_PAT",
    api_base="https://api.clarifai.com/v2/ext/openai/v1",
    messages=[{"role": "user", "content": "What is the weather in Paris today?"}],
    tools=tools,
)

print(response.choices[0].message.tool_calls)
© 2025 Clarifai, Inc. All rights reserved