Build RAG Apps
Use the Clarifai SDKs to build RAG Applications
In the field of text generation, Retrieval-Augmented Generation (RAG) enhances the capabilities of Large Language Models (LLMs) by combining information retrieval with natural language generation.
This approach addresses key limitations of LLMs by retrieving relevant data from external knowledge bases in real time, enriching responses with increased precision and contextual relevance.
Clarifai SDKs streamline the creation of RAG-based applications by minimizing the complexity of integrating retrieval and generation steps. This makes it easier for developers to build powerful, context-aware AI solutions with improved accuracy and reliability.
Click here to learn more about RAG.
Before using the Python SDK, Node.js SDK, or any of our gRPC clients, ensure they are properly installed on your machine. Refer to their respective installation guides for instructions on how to install and initialize them.
Prerequisites
-
You need a PAT (Personal Access Token) key to authenticate your connection to the Clarifai platform. You can generate the PAT key in your personal settings page by navigating to the Security section.
-
Clone the Clarifai Examples repository to get the data files required for the building RAG.
!git clone https://github.com/Clarifai/examples.git
%cd /content/examples/
To run on a local system use: cd examples/
Before you proceed install llama_index
using pip install llama-index-core==0.10.24
Initializing RAG
The first step in building a RAG-based application is initializing the RAG object. You can do this in three ways:
-
Using User ID – Automatically creates a new app.
-
Using App URL – Initializes RAG with an existing app and its data.
-
Using Workflow URL or ID – Uses a specific workflow that includes the RAG Prompter and LLM model for seamless integration.
You can set a specific version of LLM like this: https://clarifai.com/mistralai/completion/models/mistral-7B-Instruct/model_version/version_id
.
- Python SDK
- Node.js SDK
# Import the RAG module from Clarifai for conversational AI tasks
from clarifai.rag import RAG
# Set the user ID for authentication (For creating a new App)
USER_ID = 'USER_ID'
# Alternatively, initialize RAG system using an existing app's URL
APP_URL = 'APP_URL'
# Define the URL of the Mistral-7B language model
LLM_URL = 'https://clarifai.com/mistralai/completion/models/mistral-7B-Instruct'
# Define a template string for generating prompts during inference
RAG_PROMPT_TEMPLATE = "<s>[INST] Context information is below:\n{data.hits}\nGiven the context information and not prior knowledge, answer the query.\nQuery: {data.text.raw}\nAnswer: [/INST]"
# Setup a RAG object with specified parameters such as user ID, model URL, minimum score threshold, and prompt template
# Option 1: Setup a RAG object with specified parameters using the user_id to create a new app
rag_object_user = RAG.setup(
user_id=USER_ID,
pat=PAT,
llm_url=LLM_URL,
min_score=0.5,
max_results=2,
prompt_template=RAG_PROMPT_TEMPLATE
)
# Option 2: Alternatively, setup a RAG object using an existing app's URL
rag_object_app = RAG.setup(
app_url=APP_URL,
pat=PAT,
llm_url=LLM_URL,
min_score=0.5,
max_results=2,
prompt_template=RAG_PROMPT_TEMPLATE
)
# Choose which initialization method to use based on your setup:
# For new app creation:
print(rag_object_user.prompt_workflow)
# For existing app initialization:
print(rag_object_app.prompt_workflow)
Image Output

import { RAG } from "clarifai-nodejs";
// Import the RAG module from Clarifai for conversational AI tasks
// Define the URL of the Mistral-7B language model
const llmUrl =
"https://clarifai.com/mistralai/completion/models/mistral-7B-Instruct";
// Define a template string for generating prompts during inference
const ragPromptTemplate =
"<s>[INST] Context information is below:\n{data.hits}\nGiven the context information and not prior knowledge, answer the query.\nQuery: {data.text.raw}\nAnswer: [/INST]";
// Setup a RAG object with specified parameters such as user ID, model URL, minimum score threshold, and prompt template
// Option 1: Initialize using userId and create a new app
const ragObject = await RAG.setup({
authConfig: {
userId: process.env.CLARIFAI_USER_ID,
pat: process.env.CLARIFAI_PAT,
},
llmUrl,
minScore: 0.5,
maxResults: 2,
promptTemplate: ragPromptTemplate,
});
// Option 2: Alternatively, initialize using an existing app's URL
const ragObjectApp = await RAG.setup({
authConfig: {
appUrl: process.env.CLARIFAI_APP_URL,
pat: process.env.CLARIFAI_PAT,
},
llmUrl,
minScore: 0.5,
maxResults: 2,
promptTemplate: ragPromptTemplate,
});
// Choose which initialization method to use based on your setup:
// For new app creation:
console.log(ragObject.promptWorkflow);
// For existing app initialization:
console.log(ragObjectApp.promptWorkflow);
We're using Mistral-7B-Instruct as the LLM for this RAG setup, but you can choose from various LLMs available in the Clarifai Community models platform. The Clarifai SDKs let you configure parameters like min_score
, max_results
, and prompt_template
to fine-tune data retrieval.
You can also initialize RAG using a workflow created in the Clarifai Portal that includes a RAG Prompter. There are two ways to set this up — one is by providing the workflow URL as a parameter.
You should only use RAG(workflow_url)
or RAG(workflow)
when a rag workflow already exists in your app.
- Python SDK
#initialize RAG using workflow URL
WORKFLOW_URL = 'workflow_URL'
rag_object_from_url = RAG(workflow_url = WORKFLOW_URL)
Output
2024-05-08 12:21:25 INFO clarifai.rag.rag: rag.py:43
workflow_url:https://clarifai.com/8tzpjy1a841y/unst-clf/workflows/rag
INFO:clarifai.rag.rag:workflow_url:https://clarifai.com/8tzpjy1a841y/unst-clf/workflows/rag
The next option is to pass workflow_id
parameter in RAG.setup()
. This will create a new workflow in your app with the defined parameters.
- Python SDK
#initialize RAG using workflow ID
USER_ID = 'user_id'
LLM_URL = 'https://clarifai.com/openai/chat-completion/models/GPT-4'
RAG_PROMPT_TEMPLATE = "Context information is below:\n{data.hits}\nGiven the context information and not prior knowledge, answer the query.\nQuery: {data.text.raw}\nAnswer: "
rag_object_gpt = RAG.setup(user_id=USER_ID,llm_url=LLM_URL, min_score=0.5, prompt_template=RAG_PROMPT_TEMPLATE,workflow_id="workflow_id")
Output
2024-05-08 12:28:09 INFO clarifai.client.app: app.py:431
Workflow created
code: SUCCESS
description: "Ok"
req_id: "73cd1b5a80ba9e7280542b6b176213fe"
INFO:clarifai.client.app:
Workflow created
code: SUCCESS
description: "Ok"
req_id: "73cd1b5a80ba9e7280542b6b176213fe"
Input
╚══ Node: rag-prompter
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ id ┃ model_type_id ┃ app_id ┃ user_id ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ prompter-rag3-def6cc6378 │ rag-prompter │ rag_app_def6cc6378 │ 8tzpjy1a841y │
└──────────────────────────┴───────────────┴────────────────────┴──────────────┘
╚══ Node: llm
┏━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━┓
┃ id ┃ model_type_id ┃ app_id ┃ user_id ┃
┡━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━┩
│ GPT-4 │ text-to-text │ chat-completion │ openai │
└───────┴───────────────┴─────────────────┴─────────┘
Dataset Upload
Next, upload your dataset—here, we're using a Vehicle Repair Manual as the RAG source. You can use the previously created RAG object to handle the upload. One key advantage of the Clarifai SDKs is that embeddings are automatically generated and stored in the vector database during upload, making the data instantly ready for retrieval.
Supported formats for upload include: DOC, PDF, plain text files, folders containing PDFs or DOCs, and URLs pointing to PDF, DOC, or text files.
- Python SDK
- Node.js SDK
FILE_PATH="RAG/data/Crawfords_Auto_Repair_Guide.txt"
rag_object.upload(file_path=FILE_PATH,chunk_size= 1024) #parameters to split the document into chunks
Output
2024-03-20 10:34:02 INFO clarifai.client.input: input.py:674
Inputs Uploaded
code: SUCCESS
description: "Ok"
details: "All inputs successfully added"
req_id: "848c5233f9d67f1904da10c33a214ff9"
INFO:clarifai.client.input:
Inputs Uploaded
code: SUCCESS
description: "Ok"
details: "All inputs successfully added"
req_id: "848c5233f9d67f1904da10c33a214ff9"
import { RAG } from "clarifai-nodejs";
import path from "path";
// Import the RAG module from Clarifai for conversational AI tasks
// Define the URL of the Mistral-7B language model
const llmUrl =
"https://clarifai.com/mistralai/completion/models/mistral-7B-Instruct";
// Define a template string for generating prompts during inference
const ragPromptTemplate =
"<s>[INST] Context information is below:\n{data.hits}\nGiven the context information and not prior knowledge, answer the query.\nQuery: {data.text.raw}\nAnswer: [/INST]";
// Setup a RAG object with specified parameters such as user ID, model URL,
// minimum score threshold, and prompt template
const ragObject = await RAG.setup({
authConfig: {
userId: process.env.CLARIFAI_USER_ID,
pat: process.env.CLARIFAI_PAT,
},
llmUrl,
minScore: 0.5,
maxResults: 2,
promptTemplate: ragPromptTemplate,
});
const filePath = path.resolve(__dirname, "../../assets/ragInput.txt");
await ragObject.upload({
filePath,
chunkSize: 1024,
});
Chat
In the final step, we are going to perform information retrieval using RAG based on the data we provided.
- Python SDK
- Node.js SDK
# Initiating a conversation with the RAG (Retrieval Augmented Generation) model object (`rag_object_gpt`).
# Sending a message containing the query "How to change brake fluid" to the model and awaiting a response.
result = rag_object.chat(messages=[{"role": "human", "content": "How to change brake fluid"}])
# Extracting the content of the response from the result.
answer = result[0]["content"]
# Printing out the response
print(answer)
Output
To change the brake fluid, you will need to follow these steps:
1. Locate the brake fluid reservoir in your vehicle. It is usually a clear plastic container with MAX and MIN markings on it.
2. Use a turkey baster or a brake fluid pump to remove the old brake fluid from the reservoir. Be careful not to spill any brake fluid on the car's paint as it can damage the finish.
3. Once the old fluid is removed, clean the reservoir with a lint-free cloth to ensure there is no contamination.
4. Refill the reservoir with new brake fluid that is recommended for your specific vehicle. Make sure to use the type of brake fluid specified in your owner's manual.
5. Slowly pour the new brake fluid into the reservoir up to the MAX marking. Avoid overfilling.
6. After filling the reservoir, you may need to bleed the brake system to remove any air bubbles. This process may vary depending on your vehicle, so it's best to consult your owner's manual or a professional mechanic for guidance.
7. Once the brake fluid is changed and the system is bled, check for any leaks or issues before driving the vehicle.
Remember, if you are not comfortable or experienced with changing brake fluid, it is recommended to have this task done by a professional mechanic. Brake fluid is a critical component of your vehicle's braking system, and proper maintenance is essential for your safety on the road.
import { RAG } from "clarifai-nodejs";
import path from "path";
// Import the RAG module from Clarifai for conversational AI tasks
// Define the URL of the Mistral-7B language model
const llmUrl =
"https://clarifai.com/mistralai/completion/models/mistral-7B-Instruct";
// Define a template string for generating prompts during inference
const ragPromptTemplate =
"<s>[INST] Context information is below:\n{data.hits}\nGiven the context information and not prior knowledge, answer the query.\nQuery: {data.text.raw}\nAnswer: [/INST]";
// Setup a RAG object with specified parameters such as user ID, model URL,
// minimum score threshold, and prompt template
const ragObject = await RAG.setup({
authConfig: {
userId: process.env.CLARIFAI_USER_ID,
pat: process.env.CLARIFAI_PAT,
},
llmUrl,
minScore: 0.5,
maxResults: 2,
promptTemplate: ragPromptTemplate,
});
const filePath = path.resolve(__dirname, "../../assets/ragInput.txt");
await ragObject.upload({
filePath,
chunkSize: 1024,
});
// Initiating a conversation with the RAG (Retrieval Augmented Generation) model object (`ragObject`).
// Sending a message containing the query "How to change brake fluid" to the model and awaiting a response.
const result = await ragObject.chat({
messages: [{ role: "human", content: "How to change brake fluid" }],
});
// Extracting the content of the response from the result.
const answer = result[1].content;
// Printing out the response
console.log(answer);
const result2 = await ragObject.chat({
messages: [
{ role: "human", content: "procedure after following the above steps" },
],
});
const answer2 = result2[1].content;
// Printing out the response
console.log(answer2);
Now let's ask questions that are related to the answer we received before so that we can be sure the RAG has understood the context properly.
- Python SDK
result=rag_object.chat(messages=[{"role":"human", "content":"procedure after following the above steps"}])
answer=result[0]["content"]
print(answer)
Output
After following the steps to drain, flush, and pressure test the cooling system as described in the text, the next procedure would be to check for any leaks in the cooling system. This can be done by inspecting the entire cooling system, including the radiator, hoses, water pump, and heater core, for any signs of leaks. If there is less pressure on the gauge after the pressure test, there is probably a leak. Additionally, the engine should be started and the temperature gauge should be monitored to ensure that the cooling system is functioning properly. If the engine overheats or the temperature gauge reads high, further diagnosis and repair may be necessary.