Model Inference

Run inferences using Clarifai SDKs

---

Clarifai's new inference technique provides an efficient, scalable, and streamlined way to perform predictions with models.

Built with a Python-first, user-centric design, this flexible approach simplifies the process of working with models — enabling users to focus more on building and iterating, and less on navigating API mechanics.

Prerequisites

Install Clarifai Packages

• Install the latest version of the Clarifai Python SDK package:

Bash

 pip install --upgrade clarifai 

• Install the latest version of the Clarifai Node.js SDK package:

Bash

 npm install clarifai-nodejs 

Get a PAT Key

You need a PAT (Personal Access Token) key to authenticate your connection to the Clarifai platform. You can generate the PAT key in your personal settings page by navigating to the Security section.

Structure of Prediction Methods

Before making a prediction with a model, it’s important to understand how its prediction methods are structured.

You can learn more about the structure of model prediction methods here.

Get Available Methods

You can list all the methods implemented in the model's configuration that are available for performing model inference.

Python SDK

import os
from clarifai.client import Model

# Set your Personal Access Token (PAT)
os.environ["CLARIFAI_PAT"] = "YOUR_PAT_HERE"

# Initialize with model URL
model = Model(url="https://clarifai.com/openai/chat-completion/models/o4-mini")

model_methods = model.available_methods()

print(model_methods)

Node.js SDK

import { Model } from "clarifai-nodejs";

const model = new Model({
  url: "https://clarifai.com/openai/chat-completion/models/o4-mini",
  authConfig: {
    pat: process.env.CLARIFAI_PAT,
  },
});

const modelMethods = await model.availableMethods();

console.log(modelMethods);

Example Output

dict_keys(['predict', 'generate', 'chat'])

Get Method Signature

You can retrieve the method signature of a specified model's method to identify all its arguments and their type annotations, which are essential for performing model inference.

A method signature defines the method's name, its input parameters (with types and default values), and the return type, helping you understand how to properly call the method.

Python SDK

import os
from clarifai.client import Model

# Set your Personal Access Token (PAT)
os.environ["CLARIFAI_PAT"] = "YOUR_PAT_HERE"

# Initialize with model URL
model = Model(url="https://clarifai.com/openai/chat-completion/models/o4-mini")

method_name = "predict" # Or, "generate", "chat", etc

method_signature = model.method_signature(method_name= method_name)

print(method_signature)

Node.js SDK

import { Model } from "clarifai-nodejs";

const model = new Model({
  url: "https://clarifai.com/openai/chat-completion/models/o4-mini",
  authConfig: {
    pat: process.env.CLARIFAI_PAT,
  },
});

const methodSignatures = await model.methodSignatures();

console.log(methodSignatures);

Example Output

def predict(prompt: str, image: data_types.Image, images: Any, chat_history: Any, max_tokens: float = 512.0, temperature: float = 1.0, top_p: float = 0.8, reasoning_effort: str = '"low"') -> str:

Generate Example Code

You can generate a sample code snippet to better understand how to perform inference using a model.

Python SDK

import os
from clarifai.client import Model

# Set your Personal Access Token (PAT)
os.environ["CLARIFAI_PAT"] = "YOUR_PAT_HERE"

# Initialize with model URL
model = Model(url="https://clarifai.com/openai/chat-completion/models/o4-mini")

model_script = model.generate_client_script()

print(model_script)

Example Output

# Clarifai Model Client Script
# Set the environment variables `CLARIFAI_DEPLOYMENT_ID` and `CLARIFAI_PAT` to run this script.
# Example usage:
from clarifai.runners.utils import data_types
import os
from clarifai.client import Model

model = Model("www.clarifai.com/openai/chat-completion/o4-mini",
    deployment_id = os.environ['CLARIFAI_DEPLOYMENT_ID'], # Only needed for dedicated deployed models
)

# Example model prediction from different model methods:

response = model.predict(prompt='What is the future of AI?', image=Image(url='https://samples.clarifai.com/metro-north.jpg'), images=None, chat_history=None, max_tokens=512.0, temperature=1.0, top_p=0.8, reasoning_effort='"low"')
print(response)

response = model.generate(prompt='What is the future of AI?', image=Image(url='https://samples.clarifai.com/metro-north.jpg'), images=None, chat_history=None, max_tokens=512.0, temperature=0.7, top_p=0.8, reasoning_effort='"low"')
for res in response:
    print(res)

response = model.chat(messages=None, max_tokens=750.0, temperature=0.7, top_p=0.8, reasoning_effort='"low"')
for res in response:
    print(res)

📄️ Inference via API

Generate predictions with models

📄️ Legacy Inference via API

Generate predictions using our older method

📄️ Inference via UI

Generate model or workflow predictions on the UI