Skip to main content

Building RAG Applications

Learn how to build a RAG application using Clarifai SDKs

In the realm of text generation, Retrieval Augmented Generation (RAG) steps up the game for Large Language Models (LLMs) by fusing information retrieval capabilities with text generation skills, tackling key drawbacks of LLMs. When presented with a query, RAG fetches relevant information from an external knowledge base, which increases precision and contextual appropriateness through the integration of this retrieved data into the input. The Clarifai SDKs allows you to create RAG-based applications with ease by reducing the number of steps in the process.

Click here to learn more about RAG.


  • Setting up the Clarifai SDKs along with PAT. Refer to the installation and configuration with the PAT token here.

Guide to get your PAT

  • Clone the Clarifai Examples repository to get the data files required for the building RAG.
!git clone
%cd /content/examples/

To run on a local system use: cd examples/


Before you proceed install llama_index using pip install llama-index-core==0.10.24

Initialising RAG

The first part of creating a RAG-based application includes setting up the RAG object. Just by setting up the RAG object, Clarifai SDKs will automatically create the app along with a prompter model and workflow containing the RAG prompter and the LLM Model.


You can set a specific version of LLM by using

# Import the RAG module from Clarifai for conversational AI tasks
from clarifai.rag import RAG

# Set the user ID for authentication

# Define the URL of the Mistral-7B language model
LLM_URL = ''

# Define a template string for generating prompts during inference
RAG_PROMPT_TEMPLATE = "<s>[INST] Context information is below:\n{data.hits}\nGiven the context information and not prior knowledge, answer the query.\nQuery: {data.text.raw}\nAnswer: [/INST]"

# Setup a RAG object with specified parameters such as user ID, model URL,
# minimum score threshold, and prompt template
rag_object = RAG.setup(user_id=USER_ID, llm_url=LLM_URL, min_score=0.5, max_results = 2, prompt_template=RAG_PROMPT_TEMPLATE)
Image Output

Here we are opting for Mistral-7B-Instruct as the LLM Model. You can choose different LLM Models for the RAG agent from Clarifai Community Models. The Clarifai SDKs also allows you to set parameters like min_score,max_results and prompt_template for retrieving relevant data.

The Clarifai SDKs also enables users to initialize RAG using a workflow you have created in the portal which should contain a RAG prompter. There are two ways you can set up RAG using workflows. In the first method, you can provide the workflow URL as a parameter,


You should only use RAG(workflow_url) or RAG(workflow) when a rag workflow already exists in your app.

#initialize  RAG using workflow URL
WORKFLOW_URL = 'workflow_URL'
rag_object_from_url = RAG(workflow_url = WORKFLOW_URL)
2024-05-08 12:21:25 INFO     clarifai.rag.rag:                                                  

The next option is to pass workflow_id parameter in RAG.setup(), this will create a new workflow in your app with the defined parameters.

#initialize  RAG using workflow ID
USER_ID = 'user_id'
LLM_URL = ''
RAG_PROMPT_TEMPLATE = "Context information is below:\n{data.hits}\nGiven the context information and not prior knowledge, answer the query.\nQuery: {data.text.raw}\nAnswer: "
rag_object_gpt = RAG.setup(user_id=USER_ID,llm_url=LLM_URL, min_score=0.5, prompt_template=RAG_PROMPT_TEMPLATE,workflow_id="workflow_id")
2024-05-08 12:28:09 INFO                                              
Workflow created
description: "Ok"
req_id: "73cd1b5a80ba9e7280542b6b176213fe"
Workflow created
description: "Ok"
req_id: "73cd1b5a80ba9e7280542b6b176213fe"

╚══ Node: rag-prompter
id ┃ model_type_id ┃ app_id ┃ user_id ┃
│ prompter-rag3-def6cc6378 │ rag-prompter │ rag_app_def6cc6378 │ 8tzpjy1a841y │
╚══ Node: llm
id ┃ model_type_id ┃ app_id ┃ user_id ┃
│ GPT-4 │ text-to-text │ chat-completion │ openai │

Dataset Upload

The next step involves uploading the dataset. In this example, we are using a Vehicle Repair Manual as data for the RAG. You can use the RAG object we created earlier for the data upload process. Now comes the perks of using Clarifai SDKs. When you upload the data the Clarifai platform will automatically generate embeddings for the inputs and store them in the vector database which makes it ready for retrieval seconds after uploading data.


Supported formats for upload are Doc, PDF, Text, Folder Containing PDF, Doc and URL of PDF,Doc, Text files.

rag_object.upload(file_path=FILE_PATH,chunk_size= 1024) #parameters to split the document into chunks
2024-03-20 10:34:02 INFO     clarifai.client.input:                                          

Inputs Uploaded


description: "Ok"

details: "All inputs successfully added"

req_id: "848c5233f9d67f1904da10c33a214ff9"


Inputs Uploaded


description: "Ok"

details: "All inputs successfully added"

req_id: "848c5233f9d67f1904da10c33a214ff9"


In the final step, we are going to perform information retrieval using RAG based on the data we provided.

# Initiating a conversation with the RAG (Retrieval Augmented Generation) model object (`rag_object_gpt`).
# Sending a message containing the query "How to change brake fluid" to the model and awaiting a response.
result =[{"role": "human", "content": "How to change brake fluid"}])

# Extracting the content of the response from the result.
answer = result[0]["content"]

# Printing out the response
To change the brake fluid, you will need to follow these steps: 

1. Locate the brake fluid reservoir in your vehicle. It is usually a clear plastic container with MAX and MIN markings on it.

2. Use a turkey baster or a brake fluid pump to remove the old brake fluid from the reservoir. Be careful not to spill any brake fluid on the car's paint as it can damage the finish.

3. Once the old fluid is removed, clean the reservoir with a lint-free cloth to ensure there is no contamination.

4. Refill the reservoir with new brake fluid that is recommended for your specific vehicle. Make sure to use the type of brake fluid specified in your owner's manual.

5. Slowly pour the new brake fluid into the reservoir up to the MAX marking. Avoid overfilling.

6. After filling the reservoir, you may need to bleed the brake system to remove any air bubbles. This process may vary depending on your vehicle, so it's best to consult your owner's manual or a professional mechanic for guidance.

7. Once the brake fluid is changed and the system is bled, check for any leaks or issues before driving the vehicle.

Remember, if you are not comfortable or experienced with changing brake fluid, it is recommended to have this task done by a professional mechanic. Brake fluid is a critical component of your vehicle's braking system, and proper maintenance is essential for your safety on the road.

Now let's ask questions that are related to the answer we received before so that we can be sure the RAG has understood the context properly.[{"role":"human", "content":"procedure after following the above steps"}])


 After following the steps to drain, flush, and pressure test the cooling system as described in the text, the next procedure would be to check for any leaks in the cooling system. This can be done by inspecting the entire cooling system, including the radiator, hoses, water pump, and heater core, for any signs of leaks. If there is less pressure on the gauge after the pressure test, there is probably a leak. Additionally, the engine should be started and the temperature gauge should be monitored to ensure that the cooling system is functioning properly. If the engine overheats or the temperature gauge reads high, further diagnosis and repair may be necessary.