Building RAG with DSPy
Learn how to build RAG with DSPy and Clarifai Python SDK
RAG systems combine two key functionalities: information retrieval and text generation. When you ask a question, RAG first retrieves relevant information (context) from a source like a document or database. Then, it uses that context to generate a well-informed and accurate answer. DSPy acts as your mission control for building RAG systems. It provides essential tools like modules and signatures to design and execute your program efficiently. Click here to learn more about RAG.
Prerequisites
- Setting up the Clarifai Python SDK along with PAT. Refer to the installation and configuration with the PAT token here.
Guide to get your PAT
- Clone the Clarifai Examples repository to get the data files required for this example.
!git clone https://github.com/Clarifai/examples.git
%cd /content/examples/
- Install the required packages.
!pip install clarifai
!pip install langchain
!pip install dspy-ai
Initialization
The first part of creating a DSPy-Clarifai application is to set some initial fields. The variables should be configured correctly for authentication and model access.
- Python
# For the demo, we are using the "llama2-70b-chat" model from Clarifai
MODEL_URL = "https://clarifai.com/meta/Llama-2/models/llama2-70b-chat"
# PAT (Personal Access Token) provided by Clarifai for authentication
PAT = "CLARIFAI_PAT"
# User ID for authentication
USER_ID = "YOUR_ID"
# App ID for authentication
APP_ID = "YOUR_APP"
Here we are opting for llama2-70b-chat as the LLM Model. You can choose different LLM Models for the DSPy from Clarifai Community Models.
Data Ingestion
To use Clarifai as a retriever, the straightforward approach involves ingesting documents directly into the Clarifai app, which functions as your vector database. This enables easy retrieval of similar documents based on their vectors. To streamline this ingestion process, we've integrated the Clarifai vector database into our workflow. For this task, we will be using the Vehicle Repair Manual as data for DSPy.
Langchain is required only for the data ingestion step, which you can skip if data has been already ingested through alternate methods.
- Python
# Import necessary modules from LangChain for text splitting, document loading, and Clarifai vector storage
from langchain.text_splitter import CharacterTextSplitter
from langchain.document_loaders import TextLoader
from langchain.vectorstores import Clarifai as clarifaivectorstore
# Initialize a TextLoader object with the path to the text file containing documents to ingest
loader = TextLoader("/content/examples/RAG/data/Crawfords_Auto_Repair_Guide.txt") # Replace with your file path
# Load documents using the loader
documents = loader.load()
# Set up a CharacterTextSplitter to split documents into smaller chunks for efficient processing
text_splitter = CharacterTextSplitter(chunk_size=1024, chunk_overlap=200)
# Split documents into smaller chunks
docs = text_splitter.split_documents(documents)
# Create a vector database using the Clarifai vector store
clarifai_vector_db = clarifaivectorstore.from_documents(
user_id=USER_ID, # User ID for authentication
app_id=APP_ID, # App ID for authentication
documents=docs, # Split documents
pat=PAT # Personal Access Token (PAT) for authentication
)
Output
2024-04-09 13:14:16 INFO clarifai.client.input: input.py:687
Inputs Uploaded
code: SUCCESS
description: "Ok"
details: "All inputs successfully added"
req_id: "b5030582b50f20e6e78c52513cfa11dc"
INFO:clarifai.client.input:
Inputs Uploaded
code: SUCCESS
description: "Ok"
details: "All inputs successfully added"
req_id: "b5030582b50f20e6e78c52513cfa11dc"
2024-04-09 13:14:18 INFO clarifai.client.input: input.py:687
Inputs Uploaded
code: SUCCESS
description: "Ok"
details: "All inputs successfully added"
req_id: "44ccf3fbb5b8932868a107658b5ce18e"
INFO:clarifai.client.input:
Inputs Uploaded
code: SUCCESS
description: "Ok"
details: "All inputs successfully added"
req_id: "44ccf3fbb5b8932868a107658b5ce18e"
2024-04-09 13:14:20 INFO clarifai.client.input: input.py:687
Inputs Uploaded
code: SUCCESS
description: "Ok"
details: "All inputs successfully added"
req_id: "b8364c5c2a3a4b8d477f99b1028b6fec"
INFO:clarifai.client.input:
Inputs Uploaded
code: SUCCESS
description: "Ok"
details: "All inputs successfully added"
req_id: "b8364c5c2a3a4b8d477f99b1028b6fec"
2024-04-09 13:14:21 INFO clarifai.client.input: input.py:687
Inputs Uploaded
code: SUCCESS
description: "Ok"
details: "All inputs successfully added"
req_id: "7c5d3751e0a79deae8012de213768932"
INFO:clarifai.client.input:
Inputs Uploaded
code: SUCCESS
description: "Ok"
details: "All inputs successfully added"
req_id: "7c5d3751e0a79deae8012de213768932"
Setup DSPy
In the next step we are going to initialize DSPy with an LLM model from the Clarifai platform, this showcases the flexibility Clarifai offers.
- Python
# Importing necessary modules
import dspy
from dspy.retrieve.clarifai_rm import ClarifaiRM
# Initializing the language model (LLM) with Clarifai
llm = dspy.Clarifai(
model=MODEL_URL, # Clarifai model URL
api_key=PAT, # Personal Access Token (PAT) for authentication
n=2, # Number of results to return
inference_params={ # Parameters for inference
"max_tokens": 100, # Maximum number of tokens per input
'temperature': 0.6 # Temperature parameter for text generation
}
)
# Initializing the retriever model with Clarifai for document retrieval
retriever_model = ClarifaiRM(
clarifai_user_id=USER_ID, # User ID for Clarifai authentication
clarfiai_app_id=APP_ID, # App ID for Clarifai authentication
clarifai_pat=PAT, # PAT for Clarifai authentication
k=2 # Number of documents to retrieve
)
# Configuring settings for DSPy
dspy.settings.configure(
lm=llm, # Language model
rm=retriever_model # Retriever model
)
Before we move on to the next section let’s do a test run,
- Python
sentence = "Fuel pump is broken"
classify = dspy.Predict('sentence -> sentiment')
print(classify(sentence=sentence).sentiment)
Output
NEGATIVE
Sentence: The car is running smoothly
Sentiment: POSITIVE
Sentence: The car is having some issues
Sentiment: NEUTRAL
Sentence: The car is amazing
Sentiment: POSITIVE
Sentence: The car is a lemon
Sentiment: NEGATIVE
RAG with DSPy
To construct a RAG module in DSPy effectively, you first need to define its signature. The signature explains the input and output fields succinctly, mapping from "question" to "answer" in a clear and intuitive manner. Once the signature is established, you proceed to create the module itself. A module in DSPy is where you put the signature into action, defining a specific functionality that compiles and generates responses based on the given queries. To begin, you construct a signature class, detailing the required input fields and the corresponding output fields. It's essential to provide comprehensive docstrings and descriptions within the class to ensure that the DSPy signature understands the context thoroughly and can compile the best prompt for the given use case. By following these steps, you can create robust and effective modules within DSPy, enabling seamless processing and response generation for various natural language processing tasks.
The GenerateAnswer class is given below,
- Python
# Defining a class named GenerateAnswer which inherits from dspy.Signature
class GenerateAnswer(dspy.Signature):
"""Think and Answer questions based on the context provided."""
# Defining input fields with descriptions
context = dspy.InputField(desc="May contain relevant facts about user query")
question = dspy.InputField(desc="User query")
# Defining output field with description
answer = dspy.OutputField(desc="Answer in one or two lines")
The RAG class is given below,
- Python
# Define a class named RAG inheriting from dspy.Module
class RAG(dspy.Module):
# Initialize the RAG class
def __init__(self):
# Call the superclass's constructor
super().__init__()
# Initialize the retrieve module
self.retrieve = dspy.Retrieve()
# Initialize the generate_answer module using ChainOfThought with GenerateAnswer
self.generate_answer = dspy.ChainOfThought(GenerateAnswer)
# Define the forward method
def forward(self, question):
# Retrieve relevant context passages based on the input question
context = self.retrieve(question).passages
# Generate an answer based on the retrieved context and the input question
prediction = self.generate_answer(context=context, question=question)
# Return the prediction as a dspy.Prediction object containing context and answer
return dspy.Prediction(context=context, answer=prediction.answer)
Chat
In the final step, we are going to perform information retrieval using a Clarifai retriever based on factual evidence.
- Python
# Define the question to ask the RAG program
my_question = "How to change the brake fluid"
# Create a RAG (Retrieval-Augmented Generation) object
Rag_obj = RAG()
# Get the prediction from the RAG model for the given question.
# This prediction includes both the context and the answer.
predict_response_llama70b = Rag_obj(my_question)
# Print the question, predicted answer, and truncated retrieved contexts.
print(f"Question: {my_question}")
print(f"Predicted Answer: {predict_response_llama70b.answer}")
print(f"Retrieved Contexts (truncated): {[c[:200] + '...' for c in predict_response_llama70b.context]}")
Output
Question: How to change the brake fluid
Predicted Answer: Check the level of the brake fluid by looking at the markings on the reservoir. If the level is low, bring it to a mechanic.
Retrieved Contexts (truncated): ['If you don t have the owner s manual for your car then you may be able to find one\nonline by using Google or another search engine. Search the website for the make of\nthe vehicle. You could also try y...', 'If you don t have the owner s manual for your car then you may be able to find one\nonline by using Google or another search engine. Search the website for the make of\nthe vehicle. You could also try y...']