Introduction
In today’s data-driven world, businesses are increasingly turning to Large Language Models (LLMs) to unlock valuable insights from their vast troves of information. This blog post explores how to leverage the power of LLMs in conjunction with SAP HANA Cloud as Vector DB on SAP Business Technology Platform (BTP) to efficiently store, retrieve, and analyze textual data. We will delve into the core concepts of embedding generation, vector databases, and demonstrate a practical Python application showcasing these RAG capabilities.
Understanding Embeddings
At the heart of this solution lies the concept of embeddings. Embeddings are numerical representations of text, capturing the semantic meaning and relationships between words and phrases. By transforming text into dense vectors, we enable machines to understand and process language in a more meaningful way. LLMs, such as those offered by OpenAI, excel at generating high-quality embeddings that accurately reflect the underlying semantics of the text.
The Role of SAP HANA Cloud as Vector DB
While LLMs provide the foundation for semantic understanding, efficient storage and retrieval of these embeddings are crucial. This is where SAP HANA Cloud Vector DB comes into play. This powerful database leverages advanced indexing techniques specifically designed for vector data, enabling lightning-fast similarity searches. By storing document embeddings within HANA Vector DB, we can efficiently retrieve the most relevant information based on semantic similarity, rather than relying solely on keyword matching.
Leveraging the LangChain Framework
To streamline the development process, we will utilize the LangChain framework. LangChain provides a comprehensive suite of tools and utilities for building and interacting with LLMs, including components for:
Embedding Generation: Integrating with popular LLM providers like OpenAI to generate embeddings for your text data.Vector Database Integration: Seamlessly connecting with HANA Vector DB to store and retrieve embeddings.Retrieval-Augmented Generation (RAG): Combining the power of LLMs with external knowledge sources (like HANA Vector DB) to enhance the quality and relevance of LLM outputs.
Tech Stack Architecture
Building a Python Application
Create an Application with Cloud Foundry Python Buildpack
Pre-requisites: Create SAP HANA instance in BTP (I used trial account) and create a Dev space
Tools : VSCode
Python:3.12
Libraries : langchain,hdbcli,langchain-community,pypdf2,openai
Embedding : openai embedding
Let’s outline the key steps involved in building a Python application that leverages these technologies:
Step 1 : Create python application
Section 1.1: Loading the PDF and Creating Chunks
def process_pdf(file_path):
# Read PDF
reader = PdfReader(file_path)
text = “”
for page in reader.pages:
text += page.extract_text()
# Split text into chunks
text_splitter = RecursiveCharacterTextSplitter(
chunk_size=500,
chunk_overlap=50,
length_function=len,
is_separator_regex=False
)
chunks = text_splitter.split_text(text)
return chunks
This section focuses on preparing the textual data for further processing.
process_pdf function: This function takes the file path of a PDF document as input.Read PDF: It uses the PdfReader class (likely from an external library) to read the content of the PDF.Extract Text: The code iterates through each page of the PDF and extracts the text using the extract_text() method. All the extracted text is concatenated into a single string.Splitting into Chunks: Here, a RecursiveCharacterTextSplitter class (likely from LangChain) is used to split the extracted text into smaller chunks. This is done for two reasons:LLM Processing Limits: LLMs often have limitations on the amount of text they can process at once. Splitting into chunks ensures the text stays within these limits.Improved Efficiency: Processing smaller chunks can be more efficient for both the LLM and the overall application.
The function returns a list of text chunks after processing the PDF.
Section 1.2: Creating HANA DB Connection
def store_in_hana(chunks, filename):
# Initialize HANA connection
HANA_CONNECTION = dbapi.connect(
address=”xxxx.hana.trial-us10.hanacloud.ondemand.com”,
port=”443″,
user=os.environ.get(‘HANA_USER’),
password=os.environ.get(‘HANA_PASSWORD’),
autocommit=True,
sslValidateCertificate=False
)
This section establishes a connection to the SAP HANA Cloud Vector DB instance.
store_in_hana function: This function takes the list of text chunks and the original filename of the PDF as input.HANA Connection: It uses the dbapi.connect function to establish a connection with the HANA database. The connection details like address, port, username, and password are provided (likely retrieved from environment variables for security).
Note: Replace the placeholder values with your actual HANA Cloud connection details.
Section 1.3: Embedding and Storing in HANA Cloud Vector DB
# Initialize embeddings model
embeddings = OpenAIEmbeddings()
# Initialize vector store
vectorstore = HanaDB(
connection=HANA_CONNECTION,
embedding=embeddings,
table_name=”UPLOAD_DOCUMENT_VECTORS”
)
# Create documents with metadata
texts = chunks
metadatas = [{“source”: filename, “chunk”: i} for i in range(len(chunks))]
# Add documents to vector store
vectorstore.add_texts(texts=texts, metadatas=metadatas)
return len(chunks)
This section leverages LangChain and HANA Vector DB to process the text chunks and store them efficiently.
Embedding Model: An OpenAIEmbeddings instance is created (likely from LangChain) to interact with an OpenAI LLM for generating embeddings.Vector Store: A HanaDB instance is created, specifying the HANA connection, the embedding model, and the target table name (“UPLOAD_DOCUMENT_VECTORS”) within the HANA database. This essentially creates a bridge between LangChain and HANA Vector DB.Documents and Metadata: The texts variable holds the list of text chunks.The metadatas list creates metadata entries for each chunk, including the source filename (original PDF) and a chunk identifier (index within the list).Adding Documents: The add_texts method of the vectorstore object is used to add the text chunks and their corresponding metadata
Section 1.4: Retrieval and LLM Interaction
Create a ConversationalRetrievalChain using LangChain to facilitate interaction with the LLM.When a user query is received, generate an embedding for the query.Use the HANA Vector DB to retrieve the most similar documents based on the query embedding.Pass the retrieved documents to the LLM along with the user query.The LLM will then generate a response based on both the user query and the relevant context from the retrieved documents.
Section 1.5: Create a HTML page for the UI
Create a ConversationalRetrievalChain using LangChain to facilitate interaction with the LLM.When a user query is received, generate an embedding for the query.Use the HANA Vector DB to retrieve the most similar documents based on the query embedding.Pass the retrieved documents to the LLM along with the user query.The LLM will then generate a response based on both the user query and the relevant context from the retrieved documents.
Section 1.6: Test the app locally in VS code
Run the .py file to check if the application is working locally.
Open the URL in browser locally
Step 2 : Push the application to Cloud Foundry in SAP BTP
Install the cloudfoundry-cli as per the installation instructions mentioned below .
https://github.com/cloudfoundry/cli/wiki/V8-CLI-Installation-Guide
choco install cloudfoundry-cli
Check if you can access the SAP HANA Cloud DB endpoint api using the below command
cf api https://api.cf.xxxxx.hana.ondemand.com
Login using cf login –sso to authenticate. It will create a temporary link , use that link to login and get the temporary token and use it to authenticate
In the VS code terminal go to the app folder and then use the below command to push the app to the BTP
Cf push
The app will be visible in the BTP DEV space
Launch the app by clicking on the Application Routes URL as shown below
Test the File upload application
I have used the “Installation Guide for SAP S/4HANA and SAP S/4HANA Cloud Private Edition 2023” pdf for this.
Test the RAG application by running retriever app .
The source code is available in the git repo here SAP-HANA-VectorDB-RAG
Conclusion
By combining the power of LLMs, SAP HANA Vector DB, and the LangChain framework, you can build intelligent applications that can effectively understand, process, and utilize your textual data. This approach opens up a wide range of possibilities, including:
Enhanced Customer Support: Providing more accurate and personalized support through chatbots and virtual assistants.Improved Search Capabilities: Enabling users to find relevant information more efficiently through semantic search.Intelligent Document Analysis: Extracting key insights and automating tasks such as document summarization and classification.
This blog post has provided a high-level overview of this exciting technology. By exploring and experimenting further, you can unlock the full potential of LLMs and HANA Vector DB to drive innovation and transform your business processes.
References :
https://developers.sap.com/tutorials/btp-cf-buildpacks-python-create.html
IntroductionIn today’s data-driven world, businesses are increasingly turning to Large Language Models (LLMs) to unlock valuable insights from their vast troves of information. This blog post explores how to leverage the power of LLMs in conjunction with SAP HANA Cloud as Vector DB on SAP Business Technology Platform (BTP) to efficiently store, retrieve, and analyze textual data. We will delve into the core concepts of embedding generation, vector databases, and demonstrate a practical Python application showcasing these RAG capabilities.Understanding EmbeddingsAt the heart of this solution lies the concept of embeddings. Embeddings are numerical representations of text, capturing the semantic meaning and relationships between words and phrases. By transforming text into dense vectors, we enable machines to understand and process language in a more meaningful way. LLMs, such as those offered by OpenAI, excel at generating high-quality embeddings that accurately reflect the underlying semantics of the text.The Role of SAP HANA Cloud as Vector DBWhile LLMs provide the foundation for semantic understanding, efficient storage and retrieval of these embeddings are crucial. This is where SAP HANA Cloud Vector DB comes into play. This powerful database leverages advanced indexing techniques specifically designed for vector data, enabling lightning-fast similarity searches. By storing document embeddings within HANA Vector DB, we can efficiently retrieve the most relevant information based on semantic similarity, rather than relying solely on keyword matching.Leveraging the LangChain FrameworkTo streamline the development process, we will utilize the LangChain framework. LangChain provides a comprehensive suite of tools and utilities for building and interacting with LLMs, including components for:Embedding Generation: Integrating with popular LLM providers like OpenAI to generate embeddings for your text data.Vector Database Integration: Seamlessly connecting with HANA Vector DB to store and retrieve embeddings.Retrieval-Augmented Generation (RAG): Combining the power of LLMs with external knowledge sources (like HANA Vector DB) to enhance the quality and relevance of LLM outputs. Tech Stack Architecture Building a Python ApplicationCreate an Application with Cloud Foundry Python BuildpackPre-requisites: Create SAP HANA instance in BTP (I used trial account) and create a Dev spaceTools : VSCodePython:3.12Libraries : langchain,hdbcli,langchain-community,pypdf2,openaiEmbedding : openai embeddingLet’s outline the key steps involved in building a Python application that leverages these technologies:Step 1 : Create python application Section 1.1: Loading the PDF and Creating Chunksdef process_pdf(file_path):
# Read PDF
reader = PdfReader(file_path)
text = “”
for page in reader.pages:
text += page.extract_text()
# Split text into chunks
text_splitter = RecursiveCharacterTextSplitter(
chunk_size=500,
chunk_overlap=50,
length_function=len,
is_separator_regex=False
)
chunks = text_splitter.split_text(text)
return chunksThis section focuses on preparing the textual data for further processing.process_pdf function: This function takes the file path of a PDF document as input.Read PDF: It uses the PdfReader class (likely from an external library) to read the content of the PDF.Extract Text: The code iterates through each page of the PDF and extracts the text using the extract_text() method. All the extracted text is concatenated into a single string.Splitting into Chunks: Here, a RecursiveCharacterTextSplitter class (likely from LangChain) is used to split the extracted text into smaller chunks. This is done for two reasons:LLM Processing Limits: LLMs often have limitations on the amount of text they can process at once. Splitting into chunks ensures the text stays within these limits.Improved Efficiency: Processing smaller chunks can be more efficient for both the LLM and the overall application.The function returns a list of text chunks after processing the PDF.Section 1.2: Creating HANA DB Connectiondef store_in_hana(chunks, filename):
# Initialize HANA connection
HANA_CONNECTION = dbapi.connect(
address=”xxxx.hana.trial-us10.hanacloud.ondemand.com”,
port=”443″,
user=os.environ.get(‘HANA_USER’),
password=os.environ.get(‘HANA_PASSWORD’),
autocommit=True,
sslValidateCertificate=False
)This section establishes a connection to the SAP HANA Cloud Vector DB instance.store_in_hana function: This function takes the list of text chunks and the original filename of the PDF as input.HANA Connection: It uses the dbapi.connect function to establish a connection with the HANA database. The connection details like address, port, username, and password are provided (likely retrieved from environment variables for security).Note: Replace the placeholder values with your actual HANA Cloud connection details.Section 1.3: Embedding and Storing in HANA Cloud Vector DB # Initialize embeddings model
embeddings = OpenAIEmbeddings()
# Initialize vector store
vectorstore = HanaDB(
connection=HANA_CONNECTION,
embedding=embeddings,
table_name=”UPLOAD_DOCUMENT_VECTORS”
)
# Create documents with metadata
texts = chunks
metadatas = [{“source”: filename, “chunk”: i} for i in range(len(chunks))]
# Add documents to vector store
vectorstore.add_texts(texts=texts, metadatas=metadatas)
return len(chunks)This section leverages LangChain and HANA Vector DB to process the text chunks and store them efficiently.Embedding Model: An OpenAIEmbeddings instance is created (likely from LangChain) to interact with an OpenAI LLM for generating embeddings.Vector Store: A HanaDB instance is created, specifying the HANA connection, the embedding model, and the target table name (“UPLOAD_DOCUMENT_VECTORS”) within the HANA database. This essentially creates a bridge between LangChain and HANA Vector DB.Documents and Metadata: The texts variable holds the list of text chunks.The metadatas list creates metadata entries for each chunk, including the source filename (original PDF) and a chunk identifier (index within the list).Adding Documents: The add_texts method of the vectorstore object is used to add the text chunks and their corresponding metadata Section 1.4: Retrieval and LLM InteractionCreate a ConversationalRetrievalChain using LangChain to facilitate interaction with the LLM.When a user query is received, generate an embedding for the query.Use the HANA Vector DB to retrieve the most similar documents based on the query embedding.Pass the retrieved documents to the LLM along with the user query.The LLM will then generate a response based on both the user query and the relevant context from the retrieved documents.Section 1.5: Create a HTML page for the UI Create a ConversationalRetrievalChain using LangChain to facilitate interaction with the LLM.When a user query is received, generate an embedding for the query.Use the HANA Vector DB to retrieve the most similar documents based on the query embedding.Pass the retrieved documents to the LLM along with the user query.The LLM will then generate a response based on both the user query and the relevant context from the retrieved documents.Section 1.6: Test the app locally in VS code Run the .py file to check if the application is working locally. Open the URL in browser locallyStep 2 : Push the application to Cloud Foundry in SAP BTPInstall the cloudfoundry-cli as per the installation instructions mentioned below .https://github.com/cloudfoundry/cli/wiki/V8-CLI-Installation-Guidechoco install cloudfoundry-cliCheck if you can access the SAP HANA Cloud DB endpoint api using the below commandcf api https://api.cf.xxxxx.hana.ondemand.comLogin using cf login –sso to authenticate. It will create a temporary link , use that link to login and get the temporary token and use it to authenticateIn the VS code terminal go to the app folder and then use the below command to push the app to the BTPCf pushThe app will be visible in the BTP DEV spaceLaunch the app by clicking on the Application Routes URL as shown belowTest the File upload applicationI have used the “Installation Guide for SAP S/4HANA and SAP S/4HANA Cloud Private Edition 2023” pdf for this.Test the RAG application by running retriever app .The source code is available in the git repo here SAP-HANA-VectorDB-RAG ConclusionBy combining the power of LLMs, SAP HANA Vector DB, and the LangChain framework, you can build intelligent applications that can effectively understand, process, and utilize your textual data. This approach opens up a wide range of possibilities, including:Enhanced Customer Support: Providing more accurate and personalized support through chatbots and virtual assistants.Improved Search Capabilities: Enabling users to find relevant information more efficiently through semantic search.Intelligent Document Analysis: Extracting key insights and automating tasks such as document summarization and classification.This blog post has provided a high-level overview of this exciting technology. By exploring and experimenting further, you can unlock the full potential of LLMs and HANA Vector DB to drive innovation and transform your business processes.References :https://developers.sap.com/tutorials/btp-cf-buildpacks-python-create.htmlhttps://community.sap.com/t5/technology-blogs-by-sap/a-journey-into-retrieval-augmented-generation-rag-on-sap-btp/ba-p/13580653 Read More Technology Blogs by Members articles
#SAP
#SAPTechnologyblog
