Inference via API

Generate predictions using your deployed models

Our new inference technique provides an efficient, scalable, and streamlined way to perform predictions with models.

Built with a Python-first, user-centric design, this flexible approach simplifies the process of working with models — enabling users to focus more on building and iterating, and less on navigating API mechanics.

tip

Click here for additional examples on how to perform model predictions using various SDKs — such as the Clarifai SDK, OpenAI client, and LiteLLM. The examples demonstrate various model types and include both streaming and non-streaming modes, as well as tool calling capabilities.

Prerequisites

Install Clarifai Packages

Install the latest version of the Clarifai Python SDK package:

Bash

 pip install --upgrade clarifai 

Install the latest version of the Clarifai Node.js SDK package:

Bash

 npm install clarifai-nodejs 

Get a PAT Key

You need a PAT (Personal Access Token) key to authenticate your connection to the Clarifai platform. You can generate the PAT key in your personal settings page by navigating to the Security section.

You can then set the PAT as an environment variable using CLARIFAI_PAT, in which case you don’t need to define it explicitly in your code.

Unix-Like Systems
Windows

 export CLARIFAI_PAT=YOUR_PERSONAL_ACCESS_TOKEN_HERE 

 set CLARIFAI_PAT=YOUR_PERSONAL_ACCESS_TOKEN_HERE 

tip

On Windows, the Clarifai Python SDK expects a HOME environment variable, which isn’t set by default. To ensure compatibility with file paths used by the SDK, set HOME to the value of your USERPROFILE. You can set it in your Command Prompt this way: set HOME=%USERPROFILE%.

Structure of Prediction Methods

Before making a prediction with a model, it’s important to understand how its prediction methods are structured.

You can learn more about the structure of model prediction methods here.

Initializing the Model Client

Python SDK
Node.js SDK

from clarifai.client import Model

# Initialize with explicit IDs
model = Model(user_id="model_user_id", app_id="model_app_id", model_id="model_id")

# Or initialize with model URL
model = Model(url="https://clarifai.com/model_user_id/model_app_id/models/model_id")
  

import { Model } from "clarifai-nodejs";

const model = new Model({
    modelId: "model_id",
    modelUserAppId: {
      userId: "model_user_id",
      appId: "model_app_id",
    },
    authConfig: {
      pat: "pat",
    },
  });

Get Available Methods

You can list all the methods implemented in the model's configuration that are available for performing model inference.

Python SDK
Node.js SDK

import os
from clarifai.client import Model

# Set your Personal Access Token (PAT)
os.environ["CLARIFAI_PAT"] = "YOUR_PAT_HERE"

# Initialize with model URL
model = Model(url="https://clarifai.com/openai/chat-completion/models/o4-mini")

model_methods = model.available_methods()

print(model_methods)

import { Model } from "clarifai-nodejs";

const model = new Model({
  url: "https://clarifai.com/openai/chat-completion/models/o4-mini",
  authConfig: {
    pat: process.env.CLARIFAI_PAT,
  },
});

const modelMethods = await model.availableMethods();

console.log(modelMethods);

Example Output

dict_keys(['predict', 'generate', 'chat'])

Get Method Signature

You can retrieve the method signature of a specified model's method to identify all its arguments and their type annotations, which are essential for performing model inference.

A method signature defines the method's name, its input parameters (with types and default values), and the return type, helping you understand how to properly call the method.

Python SDK
Node.js SDK

import os
from clarifai.client import Model

# Set your Personal Access Token (PAT)
os.environ["CLARIFAI_PAT"] = "YOUR_PAT_HERE"

# Initialize with model URL
model = Model(url="https://clarifai.com/openai/chat-completion/models/o4-mini")

method_name = "predict" # Or, "generate", "chat", etc

method_signature = model.method_signature(method_name= method_name)

print(method_signature)

import { Model } from "clarifai-nodejs";

const model = new Model({
  url: "https://clarifai.com/openai/chat-completion/models/o4-mini",
  authConfig: {
    pat: process.env.CLARIFAI_PAT,
  },
});

const methodSignatures = await model.methodSignatures();

console.log(methodSignatures);

Example Output

def predict(prompt: str, image: data_types.Image, images: Any, chat_history: Any, max_tokens: float = 512.0, temperature: float = 1.0, top_p: float = 0.8, reasoning_effort: str = '"low"') -> str:

Generate Example Code

You can generate a sample code snippet to better understand how to perform inference using a model.

Python SDK

import os
from clarifai.client import Model

# Set your Personal Access Token (PAT)
os.environ["CLARIFAI_PAT"] = "YOUR_PAT_HERE"

# Initialize with model URL
model = Model(url="https://clarifai.com/openai/chat-completion/models/o4-mini")

model_script = model.generate_client_script()

print(model_script)

Example Output

# Clarifai Model Client Script
# Set the environment variables `CLARIFAI_DEPLOYMENT_ID` and `CLARIFAI_PAT` to run this script.
# Example usage:
from clarifai.runners.utils import data_types
import os
from clarifai.client import Model

model = Model("www.clarifai.com/openai/chat-completion/o4-mini",
    deployment_id = os.environ['CLARIFAI_DEPLOYMENT_ID'], # Only needed for dedicated deployed models
)

# Example model prediction from different model methods:

response = model.predict(prompt='What is the future of AI?', image=Image(url='https://samples.clarifai.com/metro-north.jpg'), images=None, chat_history=None, max_tokens=512.0, temperature=1.0, top_p=0.8, reasoning_effort='"low"')
print(response)

response = model.generate(prompt='What is the future of AI?', image=Image(url='https://samples.clarifai.com/metro-north.jpg'), images=None, chat_history=None, max_tokens=512.0, temperature=0.7, top_p=0.8, reasoning_effort='"low"')
for res in response:
    print(res)

response = model.chat(messages=None, max_tokens=750.0, temperature=0.7, top_p=0.8, reasoning_effort='"low"')
for res in response:
    print(res)

Set up a deployment

To use Clarifai’s Compute Orchestration capabilities, ensure your model is deployed, as described previously. Then, configure the deployment_id parameter — alternatively, you can specify compute_cluster_id and nodepool_id. If none of these are set, the prediction will default to the Clarifai Shared deployment type.

model = Model(
    url="MODEL_URL_HERE",  
    deployment_id="DEPLOYMENT_ID_HERE",
    # Or, set cluster and nodepool 
    # compute_cluster_id = "COMPUTE_CLUSTER_ID_HERE",
    # nodepool_id = "NODEPOOL_ID_HERE"
)

Unary-Unary Predict Call

This is the simplest form of prediction: a single input is sent to the model, and a single response is returned. It’s ideal for quick, non-streaming tasks, such as classifying an image or analyzing a short piece of text.

NOTE: Streaming means that the response is streamed back token by token, rather than waiting for the entire completion to be generated before returning. This is useful for building interactive applications where you want to display the response as it's being generated.

Text Inputs

Here is an example of a model signature configured on the server side for handling text inputs:

Python

@ModelClass.method
def predict(self, prompt: str = "") -> str:

Here’s how you can make a corresponding unary-unary predict call from the client side:

Python SDK
Node.js SDK

import os
from clarifai.client import Model

# Set your Personal Access Token (PAT)
os.environ["CLARIFAI_PAT"] = "YOUR_PAT_HERE"

# Initialize with model URL
model = Model(
    url="https://clarifai.com/openai/chat-completion/models/o4-mini",
    # deployment_id="DEPLOYMENT_ID_HERE"
)

response = model.predict("What is photosynthesis?")
# Or
# response = model.predict(prompt="What is photosynthesis?")

print(response)

import { Model } from "clarifai-nodejs";

const model = new Model({
  url: "https://clarifai.com/openai/chat-completion/models/o4-mini",
  authConfig: {
    pat: process.env.CLARIFAI_PAT,
  },
});

const response = await model.predict({
  // see available methodNames using model.availableMethods()
  methodName: "predict",
  prompt: "What is photosynthesis?",
});

console.log(JSON.stringify(response));

// get response data from the response object
Model.getOutputDataFromModelResponse(response);

Example Output

Photosynthesis is the process by which certain organisms—primarily plants, algae, and some bacteria—convert light energy (usually from the sun) into chemical energy stored in sugars. In essence, these organisms capture carbon dioxide (CO₂) from the air and water (H₂O) from the soil, then use sunlight to drive a series of reactions that produce oxygen (O₂) as a by-product and synthesize glucose (C₆H₁₂O₆) or related carbohydrates.

Key points:

1. Light absorption
   • Chlorophyll and other pigments in chloroplasts (in plants and algae) absorb photons, elevating electrons to higher energy states.

2. Light-dependent reactions (in thylakoid membranes)
   • Convert light energy into chemical energy in the form of ATP and NADPH.
   • Split water molecules, releasing O₂.

3. Calvin cycle (light-independent reactions, in the stroma)
   • Use ATP and NADPH to fix CO₂ into organic molecules.
   • Produce glyceraldehyde-3-phosphate (G3P), which can be converted into glucose and other carbs.

Overall simplified equation:
6 CO₂ + 6 H₂O + light energy → C₆H₁₂O₆ + 6 O₂

Importance:
• Generates the oxygen we breathe.
• Forms the base of most food chains by producing organic matter.
• Plays a critical role in the global carbon cycle and helps mitigate CO₂ in the atmosphere.

Image Inputs

Image-to-Text

Here is an example of a model signature configured on the server side for handling image inputs:

Python

@ModelClass.method
def predict(self, image: Image) -> str:

Here’s how you can make a corresponding unary-unary predict call from the client side:

Python SDK
Node.js SDK

import os
from clarifai.client import Model
from clarifai.runners.utils.data_types import Image

# Set your Personal Access Token (PAT)
os.environ["CLARIFAI_PAT"] = "YOUR_PAT_HERE"

# Initialize with model URL
model = Model(
    url="https://clarifai.com/openai/chat-completion/models/o4-mini",
    # deployment_id="DEPLOYMENT_ID_HERE"
)

response = model.predict(
    prompt="Describe the image",
    image=Image(url="https://samples.clarifai.com/cat1.jpeg") 
)

print(response)

"""
# --- Predict using an image uploaded from a local machine ---

# 1. Specify the path to your local image file
local_image_path = "path/to/your/image.jpg"  # Replace with the actual path to your image

# 2. Read the image file into bytes
with open(local_image_path, "rb") as f:
    image_bytes = f.read()

response = model.predict(
    prompt="Describe the image",
    # Provide Image as bytes
    image=Image(bytes=image_bytes)
)

print(response)

# You can also convert a Pillow (PIL) Image object into a Clarifai Image data type 
# image=Image.from_pil(pil_image)

"""

import { Model } from "clarifai-nodejs";

const model = new Model({
  url: "https://clarifai.com/openai/chat-completion/models/o4-mini",
  authConfig: {
    pat: process.env.CLARIFAI_PAT,
  },
});

const response = await model.predict({
  // see available methodNames using model.availableMethods()
  methodName: "predict",
  prompt: "Describe the image",
  image: {
    url: "https://samples.clarifai.com/cat1.jpeg",
  },
});

console.log(JSON.stringify(response));

// get response data from the response object
Model.getOutputDataFromModelResponse(response);

Example Output

The image shows a young ginger tabby cat lying on its side against what looks like a rough, earth-toned wall. Its coat is a warm orange with classic darker orange stripe markings. The cat’s front paw is tucked in, and its head rests on the surface below, with its large amber eyes gazing directly toward the viewer. The lighting is soft, highlighting the cat’s whiskers, ear fur, and the texture of its velvety coat. Overall, the scene feels calm and slightly curious, as if the cat has paused mid-nap to watch something interesting.

Visual Segmentation

Note: The following visual segmentation examples use Matplotlib, Pillow, and NumPy. You can install them by running: pip install matplotlib Pillow numpy.

Example 1

Here is an example of a model signature configured on the server side for automatic mask generation:

Python

@ModelClass.method
def segment_anything(image: data_types.Image) -> List[data_types.Region]:

Here’s how to make a corresponding unary-unary predict call from the client side to generate masks for all objects in a given image.

Python SDK

import os
import matplotlib.pyplot as plt
from clarifai.runners.utils.data_types import Image
from clarifai.client import Model
from PIL import Image as PILImage

# Set your PAT
os.environ["CLARIFAI_PAT"] = "YOUR_PAT_HERE"

# Initialize the model 
model = Model(
    url="https://clarifai.com/meta/segment-anything/models/sam2_1-hiera-base-plus",
    #deployment_id="DEPLOYMENT_ID_HERE"
)

# Load input image from URL
input_image = Image(url="https://samples.clarifai.com/cat1.jpeg")
#Or, load from local file: input_image = Image.from_pil(PILImage.open("path/to/image.png"))

# Run segmentation: returns a list of Region objects
regions = model.segment_anything(image=input_image)

# Loop through each Region, extract its mask, and display it
for idx, region in enumerate(regions):
    mask = region.mask
    #mask = Image.from_proto(region.mask.proto)  # Alternative low-level access
    # region.mask is a Clarifai Image object; convert it to a NumPy array for visualization
    mask_array = mask.to_numpy()

    # Plot the mask with matplotlib
    plt.figure(figsize=(5, 5))
    plt.imshow(mask_array, cmap='gray')
    plt.title(f"Mask {idx + 1}")
    plt.axis('off')
    plt.show()

    # Print progress
    print(f"Processed mask {idx + 1} out of {len(regions)}")

    # Optional: do anything else with mask_array here
    # (e.g., save to disk, overlay on the original image, etc.)

Example 2

Here is an example of a model signature configured on the server side for creating masks in a given image:

Python

@ModelClass.method
def predict(image: data_types.Image, regions: List[data_types.Region], dict_inputs: data_types.JSON, round_mask: bool = False, multimask_output: bool = False, denormalize_coord: bool = True) -> List[data_types.Region]:

Here’s how to make a corresponding unary-unary predict call from the client side to generate masks using a points or boxes prompt.

Python SDK

import os
import matplotlib.pyplot as plt
from PIL import Image as PILImage
from clarifai.client import Model
from clarifai.runners.utils.data_types import Image, Region, Concept

# Set your PAT
os.environ["CLARIFAI_PAT"] = "YOUR_PAT_HERE"

# Initialize the model 
model = Model(
    url="https://clarifai.com/meta/segment-anything/models/sam2_1-hiera-base-plus",
     #deployment_id="DEPLOYMENT_ID_HERE"
)

input_image = Image(url="https://samples.clarifai.com/cat1.jpeg")
# Load input image from local file
#input_image = Image.from_pil(PILImage.open("path/to/image.png"))

# Point-based input (Region form)
point_prompt_regions = [
    Region(point=[0.1, 0.2, 0.0], concepts=[Concept(name="1", value=1.0)]),
    Region(point=[0.2, 0.3, 0.0], concepts=[Concept(name="0", value=0.0)])
]

regions = model.predict(image=input_image, regions=point_prompt_regions)

# # Optional: use dict format instead
# point_prompt_dict = dict(points=[[0.1, 0.2], [0.2, 0.3]], labels=[1, 0])
# regions = model.predict(image=input_image, dict_inputs=point_prompt_dict)

# Box-based input (Region form)
# box_prompt_regions = [Region(box=[0.1, 0.2, 0.3, 0.4])]
# regions = model.predict(image=input_image, regions=box_prompt_regions)

# # Optional: use dict format instead
# box_prompt_dict = dict(box=[0.1, 0.2, 0.3, 0.4])
# regions = model.predict(image=input_image, dict_inputs=box_prompt_dict)

# Visualize each predicted mask
for idx, region in enumerate(regions):
    mask = region.mask
    #mask = Image.from_proto(region.mask.proto)  # Alternative low-level access
    # region.mask is a Clarifai Image object; convert it to a NumPy array for visualization
    mask_array = mask.to_numpy()

    # Plot the mask with matplotlib
    plt.figure(figsize=(5, 5))
    plt.imshow(mask_array, cmap='gray')
    plt.title(f"Mask {idx + 1}")
    plt.axis('off')
    plt.show()

     # Print progress
    print(f"Processed mask {idx + 1} out of {len(regions)}")

    # Optional: do anything else with mask_array here
    # (e.g., save to disk, overlay on the original image, etc.)

tip

Click here to explore how to make predictions with other data types.

Unary-Stream Predict Call

This call sends a single input to the model but returns a stream of responses. This is especially useful for tasks that produce multiple outputs from one input, such as generating text completions or progressive predictions from a prompt.

Text Inputs

Here is an example of a model signature configured on the server side for handling text inputs:

Python

@ModelClass.method
def generate(self, prompt: str) -> Iterator[str]:

Here’s how you can make a corresponding unary-stream predict call from the client side:

Python SDK
Node.js SDK

import os

from clarifai.client import Model

# Set your Personal Access Token (PAT)
os.environ["CLARIFAI_PAT"] = "YOUR_PAT_HERE"

# Initialize with model URL
model = Model(
    url="https://clarifai.com/openai/chat-completion/models/o4-mini",
    # deployment_id="DEPLOYMENT_ID_HERE"
)

response_stream = model.generate(
    prompt="Explain quantum computing in simple terms"  
)

for text_chunk in response_stream:
    print(text_chunk, end="", flush=True)

"""
# --- Load prompt text from URL ---

prompt_from_url = requests.get("https://samples.clarifai.com/featured-models/redpajama-economic.txt") # Remember to import requests
prompt_text = prompt_from_url.text.strip()

response_stream = model.generate(
    prompt=prompt_text
)

for text_chunk in response_stream:
    print(text_chunk, end="", flush=True)

"""

import { Model } from "clarifai-nodejs";

const model = new Model({
  url: "https://clarifai.com/openai/chat-completion/models/o4-mini",
  authConfig: {
    pat: process.env.CLARIFAI_PAT,
  },
});

const responseStream = model.generate({
  // see available methodNames using model.availableMethods()
  methodName: "generate",
  prompt: "what is photosynthesis?",
});

for await (const response of responseStream) {
  console.log(JSON.stringify(response));

  // get response data from the response object
  Model.getOutputDataFromModelResponse(response);
}

Image Inputs

Here is an example of a model signature configured on the server side for handling image inputs:

Python

@ModelClass.method
def generate(self, image: Image) -> Iterator[str]:

Here’s how you can make a corresponding unary-stream predict call from the client side:

Python SDK
Node.js SDK

import os
from clarifai.client import Model
from clarifai.runners.utils.data_types import Image

# Set your Personal Access Token (PAT)
os.environ["CLARIFAI_PAT"] = "YOUR_PAT_HERE"

# Initialize with model URL
model = Model(
    url="https://clarifai.com/openai/chat-completion/models/o4-mini",
    # deployment_id="DEPLOYMENT_ID_HERE"
)

response_stream = model.generate(
    prompt="Describe the image",
    image=Image(url="https://samples.clarifai.com/cat1.jpeg") 
)

for text_chunk in response_stream:
    print(text_chunk, end="", flush=True)


"""
# --- Predict using an image uploaded from a local machine ---

# 1. Specify the path to your local image file
local_image_path = "path/to/your/image.jpg"  # Replace with the actual path to your image

# 2. Read the image file into bytes
with open(local_image_path, "rb") as f:
    image_bytes = f.read()

response_stream = model.generate(
    prompt="Describe the image",
    # Provide Image as bytes
    image=Image(bytes=image_bytes)
)

for text_chunk in response_stream:
    print(text_chunk, end="", flush=True)

# You can also convert a Pillow (PIL) Image object into a Clarifai Image data type 
# image=Image.from_pil(pil_image)

"""

import { Model } from "clarifai-nodejs";

const model = new Model({
  url: "https://clarifai.com/openai/chat-completion/models/o4-mini",
  authConfig: {
    pat: process.env.CLARIFAI_PAT,
  },
});

const responseStream = await model.generate({
  // see available methodNames using model.availableMethods()
  methodName: "generate",
  prompt: "Describe the image",
  image: {
    url: "https://samples.clarifai.com/cat1.jpeg",
  },
});

for await (const response of responseStream) {
    console.log(JSON.stringify(response));
  
    // get response data from the response object
    Model.getOutputDataFromModelResponse(response);
  }

Video Inputs

Note: The following video tracking example uses Matplotlib and NumPy. You can install them by running: pip install matplotlib numpy.

Here is an example of a model signature configured on the server side for handling video inputs:

Python

@ModelClass.method
def generate(video: data_types.Video, frames: List[data_types.Frame], list_dict_inputs: List[data_types.JSON], denormalize_coord: bool = True) -> Iterator[data_types.Frame]:

Here’s how to make a corresponding unary-stream predict call from the client side to track objects in a video:

Python SDK

import os
import matplotlib.pyplot as plt
from clarifai.client import Model
from clarifai.runners.utils.data_types import Video, Region, Concept, Frame

# Set your PAT
os.environ["CLARIFAI_PAT"] = "YOUR_PAT_HERE"

# Initialize the model 
model = Model(
    url="https://clarifai.com/meta/segment-anything/models/sam2_1-hiera-base-plus",
    #deployment_id="DEPLOYMENT_ID_HERE"
)

# Load video from a URL
video = Video(url="https://samples.clarifai.com/beer.mp4")

# Or: load from local file 
# video_path = "path/to/video.mp4"
# with open(video_path, "rb") as f:
#     video = Video(bytes=f.read())

# Define frame-level prompts using Regions with track IDs
frame0 = Frame(
    regions=[
        Region(point=[0.1, 0.2, 0.0], concepts=[Concept(name="1", value=1.0)], track_id="1"),
        Region(point=[0.2, 0.3, 0.0], concepts=[Concept(name="0", value=0.0)], track_id="1"),
    ]
)
frame0.proto.frame_info.index = 0

frame1 = Frame(
    regions=[
        Region(point=[0.11, 0.22, 0.0], concepts=[Concept(name="1", value=1.0)], track_id="2"),
        Region(point=[0.22, 0.33, 0.0], concepts=[Concept(name="0", value=0.0)], track_id="2"),
    ]
)
frame1.proto.frame_info.index = 1

# Generate masks using the frame-based approach
output_frames = model.generate(video=video, frames=[frame0, frame1])

# Alternatively,  use `list_dict_inputs` instead
# frame_objs = [
#     dict(
#         points=[[0.1, 0.2], [0.2, 0.3]],
#         box=None,
#         obj_id=0,
#         labels=[1, 0],
#         frame_idx=0
#     ),
#     dict(
#         points=[[0.11, 0.22], [0.22, 0.33]],
#         box=None,
#         obj_id=1,
#         labels=[1, 0],
#         frame_idx=1
#     ),
# ]
# output_frames = model.generate(video=video, list_dict_inputs=frame_objs)

# Visualize the output masks
for frame_idx, frame in enumerate(output_frames):
    for region_idx, region in enumerate(frame.regions):
        mask = region.mask  # Clarifai Image object
        track_id = region.track_id

        # Convert mask to NumPy array
        mask_array = mask.to_numpy()

        # Show the mask
        plt.figure(figsize=(5, 5))
        plt.imshow(mask_array, cmap='gray')
        plt.title(f"Frame {frame_idx}, Region {region_idx}, Track ID: {track_id}")
        plt.axis('off')
        plt.show()

        print(f"Displayed mask for Frame {frame_idx}, Track ID: {track_id}")

Stream-Stream Predict Call

This call enables bidirectional streaming of both inputs and outputs, making it ideal for real-time applications and processing large datasets.

In this setup, multiple inputs can be continuously streamed to the model, while predictions are returned in real time. It’s especially useful for use cases like live video analysis or streaming sensor data.

Text Inputs

Here is an example of a model signature configured on the server side for handling text inputs:

Python

@ModelClass.method
def stream(self, input_iterator: Iterator[str]) -> Iterator[str]:

Here’s how you can make a corresponding stream-stream predict call from the client side:

Python SDK

import os
from clarifai.client import Model
from clarifai.runners.utils.data_types import Text

# Set your Personal Access Token (PAT)
os.environ["CLARIFAI_PAT"] = "YOUR_PAT_HERE"

# Initialize with model URL
model = Model(
    url="MODEL_URL_HERE", 
    # deployment_id="YOUR_DEPLOYMENT_ID_HERE"
)

# Create a list of input Texts to simulate a stream
input_texts = iter([
    Text(text="First input."),
    Text(text="Second input."),
    Text(text="Third input.")
])

# Call the stream method and process outputs
response_iterator = model.stream(input_texts)

# Print streamed results
print("Streaming output:\n")
for response in response_iterator:
    print(response.text)

Audio Inputs

Here is an example of a model signature configured on the server side for handling audio inputs:

Python

@ModelClass.method
def transcribe_audio(self, audio: Iterator[Audio]) -> Iterator[Text]:

Here’s how you can make a corresponding stream-stream predict call from the client side:

Python SDK

import os
from clarifai.client import Model
from clarifai.runners.utils.data_types import Audio

# Set your Personal Access Token (PAT)
os.environ["CLARIFAI_PAT"] = "YOUR_PAT_HERE"

# Initialize with model URL
model = Model(
    url="MODEL_URL_HERE",
    # deployment_id="DEPLOYMENT_ID_HERE"
)

# client-side streaming
response_stream = model.transcribe_audio(
    audio=iter(Audio(bytes=b''))
    # Or, provide audio as URL
    # audio=Audio(url="https://example.com/audio.mp3")
)

for text_chunk in response_stream:
    print(text_chunk.text, end="", flush=True)

Dynamic Batch Prediction Handling

Clarifai’s model framework seamlessly supports both single and batch predictions through a unified interface. It dynamically adapts to the input format, so no code changes are needed.

The system automatically detects the type of input provided:

If you pass a single input, it’s treated as a singleton batch;
If you pass multiple inputs as a list, they are handled as a parallel batch.

This means you can pass either a single input or a list of inputs, and the system will automatically process them appropriately — making your code cleaner and more flexible.

Image Inputs

Here is an example of a model signature configured on the server side for handling image inputs:

Python

@ModelClass.method
def predict_image(self, image: Image) -> Dict[str, float]:

Here’s how you can perform batch predictions with image inputs from the client side:

Python SDK

import os
from clarifai.client import Model
from clarifai.runners.utils.data_types import Image

# Set your Personal Access Token (PAT)
os.environ["CLARIFAI_PAT"] = "YOUR_PAT_HERE"

# Initialize with model URL
model = Model(
    url="MODEL_URL_HERE",
    # deployment_id="DEPLOYMENT_ID_HERE"
)

# Batch processing (automatically handled)
batch_results = model.predict_image([
    {"image": Image(url="https://samples.clarifai.com/cat1.jpeg")},
    {"image": Image(url="https://samples.clarifai.com/cat2.jpeg")},
])

for i, pred in enumerate(batch_results):
    print(f"Image {i+1} cat confidence: {pred['cat']:.2%}")

Text Inputs

Here is an example of a model signature configured on the server side for handling text inputs:

Python

class TextClassifier(ModelClass):
@ModelClass.method
def predict(self, text: Text) -> float:
"""Single text classification (automatically batched)"""
return self.model(text.text)

Here’s how you can perform batch predictions with text inputs from the client side:

Python SDK

import os
from clarifai.client import Model
from clarifai.runners.utils.data_types import Text

# Set your Personal Access Token (PAT)
os.environ["CLARIFAI_PAT"] = "YOUR_PAT_HERE"

# Initialize with model URL
model = Model(
    url="MODEL_URL_HERE",
    # deployment_id="DEPLOYMENT_ID_HERE"
)

# Batch prediction
batch_results = model.predict([
    {"text": Text("Positive review")},
    {"text": Text("Positive review")},
    {"text": Text("Positive review")},
])

Multimodal Predictions

You can make predictions using models that support multimodal inputs, such as a combination of images and text.

Additionally, you can configure various inference parameters to customize your prediction requests to better suit your use case.

Here is an example:

Python SDK

import os
from clarifai.client import Model
from clarifai.runners.utils.data_types import Image

# Set your Personal Access Token (PAT)
os.environ["CLARIFAI_PAT"] = "YOUR_PAT_HERE"

# Initialize model
model = Model(
    url="https://clarifai.com/openai/chat-completion/models/o4-mini",
    # deployment_id="DEPLOYMENT_ID_HERE"
)

# Perform prediction with prompt and image
result = model.predict(
    prompt="Describe this image",
    image=Image(url="https://samples.clarifai.com/metro-north.jpg"),
    max_tokens=1024    
)

# Print the prediction result
print(result)

Tool Calling

Tool calling in LLMs is a capability that allows models to autonomously decide when and how to call external tools, functions, or APIs during a conversation — based on the user’s input and the context.

You can learn more about it here.

Python SDK

from clarifai.client import Model
import os

# Set your Personal Access Token (PAT)
os.environ["CLARIFAI_PAT"] = "YOUR_PAT_HERE"

# Initialize with model URL
model = Model(url="https://clarifai.com/anthropic/completion/models/claude-sonnet-4")

# Define tools 
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get current temperature for a given location.",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "City and country e.g. Bogotá, Colombia"
                    },
                    "units": {
                        "type": "string",
                        "description": "Temperature units, e.g. Celsius or Fahrenheit",
                        "enum": ["Celsius", "Fahrenheit"]
                    }
                },
                "required": ["location"],
                "additionalProperties": False
            },
            "strict": True
        }
    }
]

response = model.generate(
    prompt="What is the temperature in Tokyo in Celsius?",
    tools=tools,
    tool_choice='auto',
    max_tokens=1024,
    temperature=0.5,
)

# Print response summary
print("Iterate or print response as needed:\n", response)

Prerequisites​

Install Clarifai Packages​

Get a PAT Key​

Structure of Prediction Methods​

Get Available Methods​

Get Method Signature​

Generate Example Code​

Unary-Unary Predict Call​

Text Inputs​

Image Inputs​

Image-to-Text​

Visual Segmentation​

Example 1​

Example 2​

Unary-Stream Predict Call​

Text Inputs​

Image Inputs​

Video Inputs​

Stream-Stream Predict Call​

Text Inputs​

Audio Inputs​

Dynamic Batch Prediction Handling​

Image Inputs​

Text Inputs​

Multimodal Predictions​

Tool Calling​

What did you think of this doc?

Prerequisites

Install Clarifai Packages

Get a PAT Key

Structure of Prediction Methods

Get Available Methods

Get Method Signature

Generate Example Code

Unary-Unary Predict Call

Text Inputs

Image Inputs

Image-to-Text

Visual Segmentation

Example 1

Example 2

Unary-Stream Predict Call

Text Inputs

Image Inputs

Video Inputs

Stream-Stream Predict Call

Text Inputs

Audio Inputs

Dynamic Batch Prediction Handling

Image Inputs

Text Inputs

Multimodal Predictions

Tool Calling