Building RAG Applications

Learn how to build a RAG application using Clarifai SDKs

In the realm of text generation, Retrieval Augmented Generation (RAG) steps up the game for Large Language Models (LLMs) by fusing information retrieval capabilities with text generation skills, tackling key drawbacks of LLMs. When presented with a query, RAG fetches relevant information from an external knowledge base, which increases precision and contextual appropriateness through the integration of this retrieved data into the input. The Clarifai SDKs allows you to create RAG-based applications with ease by reducing the number of steps in the process.

Click here to learn more about RAG.

Prerequisites

Setting up the Clarifai SDKs along with PAT. Refer to the installation and configuration with the PAT token here.

note

Guide to get your PAT

Clone the Clarifai Examples repository to get the data files required for the building RAG.

!git clone https://github.com/Clarifai/examples.git
%cd /content/examples/

info

To run on a local system use: cd examples/

note

Before you proceed install llama_index using pip install llama-index-core==0.10.24

Initialising RAG

The first part of creating a RAG-based application includes setting up the RAG object. There are three options for initialization:

Using User ID: This option will automatically create a new app.
Using App URL: This allows you to initialize RAG using an existing app, utilizing its existing data.
Using Workflow URL or ID: This enables you to initialize RAG based on a specific workflow, integrating the RAG prompter and LLM Model into the workflow.

tip

You can set a specific version of LLM by using https://clarifai.com/mistralai/completion/models/mistral-7B-Instruct/model_version/version_id.

Python
Typescript

# Import the RAG module from Clarifai for conversational AI tasks
from clarifai.rag import RAG

# Set the user ID for authentication (For creating a new App)
USER_ID = 'USER_ID'

# Alternatively, initialize RAG system using an existing app's URL
APP_URL = 'APP_URL'

# Define the URL of the Mistral-7B language model
LLM_URL = 'https://clarifai.com/mistralai/completion/models/mistral-7B-Instruct'

# Define a template string for generating prompts during inference
RAG_PROMPT_TEMPLATE = "<s>[INST] Context information is below:\n{data.hits}\nGiven the context information and not prior knowledge, answer the query.\nQuery: {data.text.raw}\nAnswer:  [/INST]"

# Setup a RAG object with specified parameters such as user ID, model URL, minimum score threshold, and prompt template

# Option 1: Setup a RAG object with specified parameters using the user_id to create a new app
rag_object_user = RAG.setup(
    user_id=USER_ID,
    pat=PAT,
    llm_url=LLM_URL,
    min_score=0.5,
    max_results=2,
    prompt_template=RAG_PROMPT_TEMPLATE
)

# Option 2: Alternatively, setup a RAG object using an existing app's URL
rag_object_app = RAG.setup(
    app_url=APP_URL,
    pat=PAT,
    llm_url=LLM_URL,
    min_score=0.5,
    max_results=2,
    prompt_template=RAG_PROMPT_TEMPLATE
)
# Choose which initialization method to use based on your setup:
# For new app creation:
print(rag_object_user.prompt_workflow)

# For existing app initialization:
print(rag_object_app.prompt_workflow)

Image Output

import { RAG } from "clarifai-nodejs";

// Import the RAG module from Clarifai for conversational AI tasks

// Define the URL of the Mistral-7B language model
const llmUrl =
  "https://clarifai.com/mistralai/completion/models/mistral-7B-Instruct";

// Define a template string for generating prompts during inference
const ragPromptTemplate =
  "<s>[INST] Context information is below:\n{data.hits}\nGiven the context information and not prior knowledge, answer the query.\nQuery: {data.text.raw}\nAnswer:  [/INST]";

// Setup a RAG object with specified parameters such as user ID, model URL, minimum score threshold, and prompt template

// Option 1: Initialize using userId and create a new app
const ragObject = await RAG.setup({
  authConfig: {
    userId: process.env.CLARIFAI_USER_ID,
    pat: process.env.CLARIFAI_PAT,
  },
  llmUrl,
  minScore: 0.5,
  maxResults: 2,
  promptTemplate: ragPromptTemplate,
});

// Option 2: Alternatively, initialize using an existing app's URL
const ragObjectApp = await RAG.setup({
  authConfig: {
    appUrl: process.env.CLARIFAI_APP_URL,
    pat: process.env.CLARIFAI_PAT,
  },
  llmUrl,
  minScore: 0.5,
  maxResults: 2,
  promptTemplate: ragPromptTemplate,
});

  // Choose which initialization method to use based on your setup:
  // For new app creation:
  console.log(ragObject.promptWorkflow);

  // For existing app initialization:
  console.log(ragObjectApp.promptWorkflow);

Here we are opting for Mistral-7B-Instruct as the LLM Model. You can choose different LLM Models for the RAG agent from Clarifai Community Models. The Clarifai SDKs also allows you to set parameters like min_score,max_results and prompt_template for retrieving relevant data.

The Clarifai SDKs also enables users to initialize RAG using a workflow you have created in the portal which should contain a RAG prompter. There are two ways you can set up RAG using workflows. In the first method, you can provide the workflow URL as a parameter,

info

You should only use RAG(workflow_url) or RAG(workflow) when a rag workflow already exists in your app.

Python

#initialize  RAG using workflow URL
WORKFLOW_URL = 'workflow_URL'
rag_object_from_url = RAG(workflow_url = WORKFLOW_URL)

Output

2024-05-08 12:21:25 INFO     clarifai.rag.rag:                                                            rag.py:43
                             workflow_url:https://clarifai.com/8tzpjy1a841y/unst-clf/workflows/rag                 
INFO:clarifai.rag.rag:workflow_url:https://clarifai.com/8tzpjy1a841y/unst-clf/workflows/rag

The next option is to pass workflow_id parameter in RAG.setup(), this will create a new workflow in your app with the defined parameters.

Python

#initialize  RAG using workflow ID
USER_ID = 'user_id'
LLM_URL = 'https://clarifai.com/openai/chat-completion/models/GPT-4'
RAG_PROMPT_TEMPLATE = "Context information is below:\n{data.hits}\nGiven the context information and not prior knowledge, answer the query.\nQuery: {data.text.raw}\nAnswer: "
rag_object_gpt = RAG.setup(user_id=USER_ID,llm_url=LLM_URL, min_score=0.5, prompt_template=RAG_PROMPT_TEMPLATE,workflow_id="workflow_id")

Output

2024-05-08 12:28:09 INFO     clarifai.client.app:                                                        app.py:431
                             Workflow created                                                                      
                             code: SUCCESS                                                                         
                             description: "Ok"                                                                     
                             req_id: "73cd1b5a80ba9e7280542b6b176213fe"                                            
                                                                                                                   
INFO:clarifai.client.app:
Workflow created
code: SUCCESS
description: "Ok"
req_id: "73cd1b5a80ba9e7280542b6b176213fe"

 Input
╚══                                Node: rag-prompter                               
    ┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
    ┃ id                       ┃ model_type_id ┃ app_id             ┃ user_id      ┃
    ┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
    │ prompter-rag3-def6cc6378 │ rag-prompter  │ rag_app_def6cc6378 │ 8tzpjy1a841y │
    └──────────────────────────┴───────────────┴────────────────────┴──────────────┘
    ╚══                       Node: llm                      
        ┏━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━┓
        ┃ id    ┃ model_type_id ┃ app_id          ┃ user_id ┃
        ┡━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━┩
        │ GPT-4 │ text-to-text  │ chat-completion │ openai  │
        └───────┴───────────────┴─────────────────┴─────────┘

Dataset Upload

The next step involves uploading the dataset. In this example, we are using a Vehicle Repair Manual as data for the RAG. You can use the RAG object we created earlier for the data upload process. Now comes the perks of using Clarifai SDKs. When you upload the data the Clarifai platform will automatically generate embeddings for the inputs and store them in the vector database which makes it ready for retrieval seconds after uploading data.

info

Supported formats for upload are Doc, PDF, Text, Folder Containing PDF, Doc and URL of PDF,Doc, Text files.

Python
Typescript

FILE_PATH="RAG/data/Crawfords_Auto_Repair_Guide.txt"
rag_object.upload(file_path=FILE_PATH,chunk_size= 1024) #parameters to split the document into chunks

Output

2024-03-20 10:34:02 INFO     clarifai.client.input:                                                    input.py:674

                             Inputs Uploaded                                                                       

                             code: SUCCESS                                                                         

                             description: "Ok"                                                                     

                             details: "All inputs successfully added"                                              

                             req_id: "848c5233f9d67f1904da10c33a214ff9"                                            

INFO:clarifai.client.input:

Inputs Uploaded

code: SUCCESS

description: "Ok"

details: "All inputs successfully added"

req_id: "848c5233f9d67f1904da10c33a214ff9"

import { RAG } from "clarifai-nodejs";
import path from "path";

// Import the RAG module from Clarifai for conversational AI tasks

// Define the URL of the Mistral-7B language model
const llmUrl =
  "https://clarifai.com/mistralai/completion/models/mistral-7B-Instruct";

// Define a template string for generating prompts during inference
const ragPromptTemplate =
  "<s>[INST] Context information is below:\n{data.hits}\nGiven the context information and not prior knowledge, answer the query.\nQuery: {data.text.raw}\nAnswer:  [/INST]";

// Setup a RAG object with specified parameters such as user ID, model URL,
// minimum score threshold, and prompt template
const ragObject = await RAG.setup({
  authConfig: {
    userId: process.env.CLARIFAI_USER_ID,
    pat: process.env.CLARIFAI_PAT,
  },
  llmUrl,
  minScore: 0.5,
  maxResults: 2,
  promptTemplate: ragPromptTemplate,
});

const filePath = path.resolve(__dirname, "../../assets/ragInput.txt");
await ragObject.upload({
  filePath,
  chunkSize: 1024,
});

Chat

In the final step, we are going to perform information retrieval using RAG based on the data we provided.

Python
Typescript

# Initiating a conversation with the RAG (Retrieval Augmented Generation) model object (`rag_object_gpt`).
# Sending a message containing the query "How to change brake fluid" to the model and awaiting a response.
result = rag_object.chat(messages=[{"role": "human", "content": "How to change brake fluid"}])

# Extracting the content of the response from the result.
answer = result[0]["content"]

# Printing out the response
print(answer)

Output

To change the brake fluid, you will need to follow these steps: 

Locate the brake fluid reservoir in your vehicle. It is usually a clear plastic container with MAX and MIN markings on it. 

Use a turkey baster or a brake fluid pump to remove the old brake fluid from the reservoir. Be careful not to spill any brake fluid on the car's paint as it can damage the finish. 

Once the old fluid is removed, clean the reservoir with a lint-free cloth to ensure there is no contamination. 

Refill the reservoir with new brake fluid that is recommended for your specific vehicle. Make sure to use the type of brake fluid specified in your owner's manual. 

Slowly pour the new brake fluid into the reservoir up to the MAX marking. Avoid overfilling. 

After filling the reservoir, you may need to bleed the brake system to remove any air bubbles. This process may vary depending on your vehicle, so it's best to consult your owner's manual or a professional mechanic for guidance. 

Once the brake fluid is changed and the system is bled, check for any leaks or issues before driving the vehicle. 

Remember, if you are not comfortable or experienced with changing brake fluid, it is recommended to have this task done by a professional mechanic. Brake fluid is a critical component of your vehicle's braking system, and proper maintenance is essential for your safety on the road.

import { RAG } from "clarifai-nodejs";
import path from "path";

// Import the RAG module from Clarifai for conversational AI tasks

// Define the URL of the Mistral-7B language model
const llmUrl =
  "https://clarifai.com/mistralai/completion/models/mistral-7B-Instruct";

// Define a template string for generating prompts during inference
const ragPromptTemplate =
  "<s>[INST] Context information is below:\n{data.hits}\nGiven the context information and not prior knowledge, answer the query.\nQuery: {data.text.raw}\nAnswer:  [/INST]";

// Setup a RAG object with specified parameters such as user ID, model URL,
// minimum score threshold, and prompt template
const ragObject = await RAG.setup({
  authConfig: {
    userId: process.env.CLARIFAI_USER_ID,
    pat: process.env.CLARIFAI_PAT,
  },
  llmUrl,
  minScore: 0.5,
  maxResults: 2,
  promptTemplate: ragPromptTemplate,
});

const filePath = path.resolve(__dirname, "../../assets/ragInput.txt");
await ragObject.upload({
  filePath,
  chunkSize: 1024,
});

// Initiating a conversation with the RAG (Retrieval Augmented Generation) model object (`ragObject`).
// Sending a message containing the query "How to change brake fluid" to the model and awaiting a response.
const result = await ragObject.chat({
  messages: [{ role: "human", content: "How to change brake fluid" }],
});

// Extracting the content of the response from the result.
const answer = result[1].content;

// Printing out the response
console.log(answer);

const result2 = await ragObject.chat({
  messages: [
    { role: "human", content: "procedure after following the above steps" },
  ],
});

const answer2 = result2[1].content;

// Printing out the response
console.log(answer2);

Now let's ask questions that are related to the answer we received before so that we can be sure the RAG has understood the context properly.

Python

result=rag_object.chat(messages=[{"role":"human", "content":"procedure after following the above steps"}])

answer=result[0]["content"]

print(answer)

Output

 After following the steps to drain, flush, and pressure test the cooling system as described in the text, the next procedure would be to check for any leaks in the cooling system. This can be done by inspecting the entire cooling system, including the radiator, hoses, water pump, and heater core, for any signs of leaks. If there is less pressure on the gauge after the pressure test, there is probably a leak. Additionally, the engine should be started and the temperature gauge should be monitored to ensure that the cooling system is functioning properly. If the engine overheats or the temperature gauge reads high, further diagnosis and repair may be necessary.

Building RAG Applications

Prerequisites​

Initialising RAG​

Dataset Upload​

Chat​

What did you think of this doc?

Prerequisites

Initialising RAG

Dataset Upload

Chat