Run Models Locally

Develop, test, and run your models directly on your own infrastructure

Local Dev Runners are a powerful feature of the Clarifai platform that let you develop, test, and execute models on your local machine.

Instead of running solely in the cloud, you can run your models anywhere that supports Python and has an internet connection — whether it's your laptop, workstation, or on-premise server.

By leveraging your own compute resources, Local Dev Runners give you full control over your development environment while maintaining seamless integration with the Clarifai API.

Your model can securely receive and process requests, just as it would in a production cloud deployment.

Here is how to run a model as a local development runner:

Bash

 clarifai model local-dev [OPTIONS] [MODEL_PATH] 

MODEL_PATH is an optional path to the model directory. If omitted, the current directory is used by default.

Use Cases for Local Dev Runners

Streamlined model development — Local Dev Runners make it easy to build and test new models directly within your local environment. You can spin up a runner on your machine, route API calls through our public cloud endpoint, and watch requests hit your model in real time. This allows you to debug, set breakpoints, return results, and validate outputs.
Leverage your own compute resources — If you have powerful hardware, you can take advantage of that local compute without relying on Clarifai's autoscaling infrastructure. Your model remains accessible through our API with full authentication, even though it runs locally.
Locally connect agents — Because Local Dev Runners execute on your chosen hardware, they can interact with local file systems, make OS-level calls, or access private data stores. With our MCP (Model Context Protocol) model type, you can give your cloud-hosted agents or any MCP-enabled clients authenticated access to your locally controlled information — regardless of their deployment location.

Prerequisites

Before you can start developing and testing your models locally with Local Dev Runners, there are a couple of things you'll need.

Build a Model

You can either create a custom model from scratch or leverage pre-trained models from external sources like Hugging Face.

If you're building your own model, follow our comprehensive step-by-step guide to get started.

You can also explore our examples repository to see models built for compatibility with the Clarifai platform.

Install Clarifai CLI

Install the latest version of the Clarifai CLI (Command Line Interface) tool. It includes built-in support for Local Dev Runners.

Bash

 pip install --upgrade clarifai 

Get a PAT Key

To authenticate your connection with the Clarifai platform, you'll need a Personal Access Token (PAT) key. You can generate the PAT key in your personal settings page by navigating to the Security section.

Create a Context

Running the local development runner relies on certain environment variables defined in your current context. The context refers to the active environment settings that determine how your commands interact with the Clarifai platform.

You can create this context using the provided default values when you run the local-dev command.

Any configurations you create locally — such as the computer cluster and app — will also be created on the Clarifai platform, making them reusable whenever you test your model with the local development runner.

tip

Click here to learn how to create and manage various aspects of your Clarifai context, including switching contexts and editing your configuration information.

These are the environment variables required to create a runner:

Variable	Description
CLARIFAI_PAT	Personal Access Token for authentication
CLARIFAI_USER_ID (`user_id`)	User ID of the account owning the model
CLARIFAI_APP_ID (`app_id`)	App ID containing the model
CLARIFAI_MODEL_ID (`model_id`)	The model ID for the model to be run locally
CLARIFAI_COMPUTE_CLUSTER_ID (`compute_cluster_id`)	Compute cluster where the Local Dev Runner will reside. Note that the `user_id` of the compute cluster must match the `user_id` of the model.
CLARIFAI_NODEPOOL_ID (`nodepool_id`)	Nodepool within the compute cluster
CLARIFAI_DEPLOYMENT (`deployment_id`)	Deployment for a model into the cluster and nodepool
CLARIFAI_RUNNER_ID (`runner_id`)	Auto-generated runner ID, created by the API and stored in the context

Run the Test

note

We'll use this method to run and test the model we created here.

To run your model with the local development runner, navigate to the directory where your custom model is located.

Then, follow these steps.

Log In

Run the following command to log in to the Clarifai platform and establish a connection.

Bash

 clarifai login

After running the login command, you'll be prompted to enter the following details to authenticate your connection:

Bash

context name (default: "default"):
user id:
personal access token value (default: "ENVVAR" to get our env var rather than config):

Context Name — You can provide a custom name for your Clarifai configuration context, or simply press Enter to use the default name, "default". This helps you manage different configurations if needed.
User ID — Enter your Clarifai user ID.
PAT — Enter your Clarifai PAT. Note that if you press Enter, and you have set the CLARIFAI_PAT environment variable, it will use that token automatically.

Start the Runner

Next, start a local development runner.

Bash

 clarifai model local-dev

If the runner doesn't detect the necessary context configurations in your environment, it will prompt you to create them using default values.

This ensures that all essential components required for Local Dev Runners are properly set up or included in your configuration context, including:

A compute cluster and nodepool configured for Local Dev Runners.

Note: This cluster is created exclusively for Local Dev Runners. It is not designed to support other tasks and lacks features like autoscaling to handle variable traffic demands, among other cloud-specific capabilities. You also cannot use other types of clusters for Local Dev Runners — only the special cluster created for this purpose is supported.

An app with a model and model version representing the local runner.
A deployment that places the model version into the designated nodepool.

Example Output

C:\#############>clarifai login
context name (default: "default"):
user id: alfrick
personal access token value (default: "ENVVAR" to get our of env var rather than config): c02f72c***************

C:\#############>clarifai model local-dev
[INFO] 14:00:01.177806 Checking setup for local development runner... |  thread=3524
[INFO] 14:00:01.179203 Current context: default |  thread=3524
[INFO] 14:00:01.189128 Current user_id: alfrick |  thread=3524
[INFO] 14:00:01.189128 Current compute_cluster_id: local-dev-compute-cluster |  thread=3524
[INFO] 14:00:02.394761 Failed to get compute cluster with ID local-dev-compute-cluster: code: CONN_DOES_NOT_EXIST
description: "Resource does not exist"
details: "ComputeCluster with ID \'local-dev-compute-cluster\' not found. Check your request fields."
req_id: "sdk-python-11.4.1-c8a4b5bab7a84ca7a91cd10fbc620c0f"
 |  thread=3524
Compute cluster not found. Do you want to create a new compute cluster alfrick/local-dev-compute-cluster? (y/n): y
[INFO] 14:00:20.755481
Compute Cluster created
code: SUCCESS
description: "Ok"
req_id: "sdk-python-11.4.1-31b3b07f08f04cbb81a50edfba30454f"
 |  thread=3524
[INFO] 14:00:20.768179 Current nodepool_id: local-dev-nodepool |  thread=3524
[INFO] 14:00:21.948659 Failed to get nodepool with ID local-dev-nodepool: code: CONN_DOES_NOT_EXIST
description: "Resource does not exist"
details: "Nodepool not found. Check your request fields."
req_id: "sdk-python-11.4.1-2e792b00b7d64521942e09f990db7e84"
 |  thread=3524
Nodepool not found. Do you want to create a new nodepool alfrick/local-dev-compute-cluster/local-dev-nodepool? (y/n): y
[INFO] 14:00:25.698797
Nodepool created
code: SUCCESS
description: "Ok"
req_id: "sdk-python-11.4.1-65d15f07e7534d44aa79a0d1a8607350"
 |  thread=3524
[INFO] 14:00:25.720799 Current app_id: local-dev-runner-app |  thread=3524
[INFO] 14:00:26.010459 Failed to get app with ID local-dev-runner-app: code: CONN_DOES_NOT_EXIST
description: "Resource does not exist"
details: "app identified by path /users/alfrick/apps/local-dev-runner-app not found"
req_id: "sdk-python-11.4.1-b3c6fcd7b4ad4c2e91f742dc31705a5b"
 |  thread=3524
App not found. Do you want to create a new app alfrick/local-dev-runner-app? (y/n): y
[INFO] 14:00:31.354436
App created
code: SUCCESS
description: "Ok"
req_id: "sdk-python-11.4.1-425accc65e1b451f9f6909d9dac25ecb"
 |  thread=3524
[INFO] 14:00:31.363481 Current model_id: local-dev-model |  thread=3524
[INFO] 14:00:33.396312 Failed to get model with ID local-dev-model: code: MODEL_DOES_NOT_EXIST
description: "Model does not exist"
details: "Model \'local-dev-model\' does not exist."
req_id: "sdk-python-11.4.1-024501ac37044ed4a4ed5a397f860cf8"
 |  thread=3524
Model not found. Do you want to create a new model alfrick/local-dev-runner-app/models/local-dev-model? (y/n): y
[INFO] 14:00:43.972640
Model created
code: SUCCESS
description: "Ok"
req_id: "sdk-python-11.4.1-052d3566144440bc83d3e5b66826a396"
 |  thread=3524
[INFO] 14:00:45.317992 No model versions found. Creating a new version for local dev runner. |  thread=3524
[INFO] 14:00:45.695709
Model Version created
code: SUCCESS
description: "Ok"
req_id: "sdk-python-11.4.1-cc9130981bea437895364a1a0debb079"
 |  thread=3524
[INFO] 14:00:45.698444 Created model version 5aaae29b6551422ea9f78c84e5f19205 |  thread=3524
[INFO] 14:00:45.698444 Current model version 5aaae29b6551422ea9f78c84e5f19205 |  thread=3524
[INFO] 14:00:45.698444 Create the local dev runner tying this
  alfrick/local-dev-runner-app/models/local-dev-model model (version: 5aaae29b6551422ea9f78c84e5f19205) to the
  alfrick/local-dev-compute-cluster/local-dev-nodepool nodepool. |  thread=3524
[INFO] 14:00:46.922695
Runner created
code: SUCCESS
description: "Ok"
req_id: "sdk-python-11.4.1-c1176211bd5e4cceadb35040f8d8d2a7"
 with id: 9dffc801b4904095a987b5c2b8508edf |  thread=3524
[INFO] 14:00:46.936701 Current runner_id: 9dffc801b4904095a987b5c2b8508edf |  thread=3524
[INFO] 14:00:47.239289 Current deployment_id: local-dev-deployment |  thread=3524
[INFO] 14:00:47.241289 Current model section of config.yaml: {'id': 'my-uploaded-model', 'user_id': 'alfrick', 'app_id': 'docs-demos', 'model_type_id': 'text-to-text'} |  thread=3524
Do you want to backup config.yaml to config.yaml.bk then update the config.yaml with the new model information? (y/n): y
[INFO] 14:00:51.122777

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
# About to start up the local dev runner in this terminal...
# Here is a code snippet to call this model once it start from another terminal:
 |  thread=3524
[INFO] 14:00:51.123778
# Clarifai Model Client Script
# Example usage:
import os

from clarifai.client import Model
from clarifai.runners.utils import data_types

model = Model.from_current_context()

# Example model prediction from different model methods:

response = model.predict(text1='""')
print(response)

response = model.generate(text1='""')
for res in response:
    print(res)

 |  thread=3524
[INFO] 14:00:51.123778 Now starting the local dev runner... |  thread=3524

Note: If the config.yaml file does not contain model information that matches the user_id, app_id, and model_id defined in your current context, it will be automatically updated to include the new model details. This ensures that the model started by the local development runner is the same one you intend to call via the API. If needed, you can back up the existing config.yaml file as config.yaml.bk.

Test with Snippet

Once the local development runner starts in your terminal, an example client code snippet is automatically generated based on the model's signature to help you test it.

Example Code Snippet

# Clarifai Model Client Script
# Example usage:
import os

from clarifai.client import Model
from clarifai.runners.utils import data_types

model = Model.from_current_context()

# Example model prediction from different model methods:

response = model.predict(text1='""')
print(response)

response = model.generate(text1='""')
for res in response:
    print(res)

If you run the generated snippet in a separate terminal, but within the same directory, you’ll receive the model’s response output.

After you're done testing, simply close the terminal running the local development runner to shut it down.

Use Cases for Local Dev Runners​

Prerequisites​

Build a Model​

Install Clarifai CLI​

Get a PAT Key​

Create a Context​

Run the Test​

Log In​

Start the Runner​

Test with Snippet​

What did you think of this doc?