diff --git a/langchain-rag-23ai/lab0/images/add-api-key-dialog1.png b/langchain-rag-23ai/lab0/images/add-api-key-dialog1.png new file mode 100644 index 0000000..0ecdfd8 Binary files /dev/null and b/langchain-rag-23ai/lab0/images/add-api-key-dialog1.png differ diff --git a/langchain-rag-23ai/lab0/images/add-api-key-dialog2.png b/langchain-rag-23ai/lab0/images/add-api-key-dialog2.png new file mode 100644 index 0000000..f890859 Binary files /dev/null and b/langchain-rag-23ai/lab0/images/add-api-key-dialog2.png differ diff --git a/langchain-rag-23ai/lab0/images/add-api-key-fingerprint.png b/langchain-rag-23ai/lab0/images/add-api-key-fingerprint.png new file mode 100644 index 0000000..5923a55 Binary files /dev/null and b/langchain-rag-23ai/lab0/images/add-api-key-fingerprint.png differ diff --git a/langchain-rag-23ai/lab0/images/add-api-key.png b/langchain-rag-23ai/lab0/images/add-api-key.png new file mode 100644 index 0000000..5e8c216 Binary files /dev/null and b/langchain-rag-23ai/lab0/images/add-api-key.png differ diff --git a/langchain-rag-23ai/lab0/images/api-configfilepreview.png b/langchain-rag-23ai/lab0/images/api-configfilepreview.png new file mode 100644 index 0000000..f845690 Binary files /dev/null and b/langchain-rag-23ai/lab0/images/api-configfilepreview.png differ diff --git a/langchain-rag-23ai/lab0/images/compartment-copy.png b/langchain-rag-23ai/lab0/images/compartment-copy.png new file mode 100644 index 0000000..b0dc5ba Binary files /dev/null and b/langchain-rag-23ai/lab0/images/compartment-copy.png differ diff --git a/langchain-rag-23ai/lab0/images/create-policy.png b/langchain-rag-23ai/lab0/images/create-policy.png new file mode 100644 index 0000000..2023f8a Binary files /dev/null and b/langchain-rag-23ai/lab0/images/create-policy.png differ diff --git a/langchain-rag-23ai/lab0/images/i-s-compartments.png b/langchain-rag-23ai/lab0/images/i-s-compartments.png new file mode 100644 index 0000000..9bafdcf Binary files /dev/null and b/langchain-rag-23ai/lab0/images/i-s-compartments.png differ diff --git a/langchain-rag-23ai/lab0/images/new-profilename.png b/langchain-rag-23ai/lab0/images/new-profilename.png new file mode 100644 index 0000000..11eb6e1 Binary files /dev/null and b/langchain-rag-23ai/lab0/images/new-profilename.png differ diff --git a/langchain-rag-23ai/lab0/images/selectapikeys.png b/langchain-rag-23ai/lab0/images/selectapikeys.png new file mode 100644 index 0000000..5a0304a Binary files /dev/null and b/langchain-rag-23ai/lab0/images/selectapikeys.png differ diff --git a/langchain-rag-23ai/lab0/images/selecttenancy.png b/langchain-rag-23ai/lab0/images/selecttenancy.png new file mode 100644 index 0000000..b81bd1a Binary files /dev/null and b/langchain-rag-23ai/lab0/images/selecttenancy.png differ diff --git a/langchain-rag-23ai/lab0/images/tenancy-ocid.png b/langchain-rag-23ai/lab0/images/tenancy-ocid.png new file mode 100644 index 0000000..29bc261 Binary files /dev/null and b/langchain-rag-23ai/lab0/images/tenancy-ocid.png differ diff --git a/langchain-rag-23ai/lab0/images/user-ocid.png b/langchain-rag-23ai/lab0/images/user-ocid.png new file mode 100644 index 0000000..5ec56e9 Binary files /dev/null and b/langchain-rag-23ai/lab0/images/user-ocid.png differ diff --git a/langchain-rag-23ai/lab0/lab0.md b/langchain-rag-23ai/lab0/lab0.md new file mode 100644 index 0000000..2f258d9 --- /dev/null +++ b/langchain-rag-23ai/lab0/lab0.md @@ -0,0 +1,117 @@ +# Lab 0: Setup to access OCI Generative AI Service + +## Introduction + +In this lab you will setup access from ADB to OCI GenAI service. + +Oracle's GenAI service is an LLM service from Oracle Cloud Infrastructure (OCI). The OCI GenAI service provides access to several LLMs that you can pick from. To enable client applications to access these services, authentication is required. This is accomplished through public and private api keys. The public key is created in the user cloud account, while the corresponding private key is stored on the server where the application runs. + +*Estimated Lab Time*: 10 minutes + +### Objectives: +* Learn the basic components of security authentication and authorization to use OCI GenAI service. + +### Prerequisites: +* You need an Oracle Cloud tenancy and the ability to grant privileges to groups + +## **Task 1:** Get your credential information to access OCI APIs +### Fingerprint and private key + +1. Open the OCI Console, click the **Profile** icon, then select **User Settings**. + + ![](images/new-profilename.png " ") + +2. The User Details page opens. Under Resources, on the lower left side of the page, select **API Keys** + + ![](images/selectapikeys.png " ") + +3. Click the **Add API Key** button + + ![](images/add-api-key.png " ") + +4. The Add API Key dialog box opens. Select option: **Generate API Key Pair** + + ![](images/add-api-key-dialog1.png " ") + +5. Click the **Download Private Key** button and save the file to your local computer. Remember the location where you saved the private key file (username-date.pem). + + ![](images/add-api-key-dialog2.png " ") + +6. Click the **Add** button in the **Add API Key** dialog. The *Configuration File Preview* dialog opens. Follow the instructions in the **Note** to copy the contents of the text box into a file. + + ![](images/api-configfilepreview.png " ") + +7. Click **Close** on the *Configuration File Preview* dialog + +## **Task 2:** Copy your Compartment OCID + +1. Go to the OCI Console and use the hamburger menu to select **Identity & Security**, then, under *Identity*, select **Compartments**. + ![](./images/i-s-compartments.png " ") + +2. The *Compartments* page is displayed and lists all of the compartments. If you are using a new trial tenancy, you will have a root compartment. Click the name of the compartment where you will running this lab. + +3. On the *Compartment details* page, click **Copy** next to the compartment OCID. + ![](./images/compartment-copy.png " ") + +4. Save the compartment OCID for use later. + +## **Task 3:** Create policy to enable access to OCI GenAI + +Create an OCI policy to enable access to OCI GenAI service. +Create a policy that will allow you to use OCI GenAI within your previously defined compartment. Make sure your policy uses the compartment where your database is deployed. The policy will be necessary for the database to interact with OCI Generative AI. + +1. From the Console, open the Navigation menu and click Identity & Security. Under Identity, click Policies. + +2. Click on Create policy and paste the following into the appropriate fields: + +Note: Slide the Show manual editor control to display the text field in order to paste the policy. + +Name: PublicGenAI + +Description: Public Gen AI Policy + +Compartment: select your own compartment + +Policy: allow any-user to manage generative-ai-family in compartment \ + +3. Click Create. + + ![](images/create-policy.png " ") + +Note: This policy allows any database in the specified compartment to access OCI Generative AI service. In a production environment, ensure your policy's scope is minimally inclusive. + +## **Task :4** Create the credentials for ADB to access OCI GenAI service + +1. Open a terminal window in your noVNC remote desktop +2. Change directory to /home/oracle/AIdemo +3. Using an editor such as vi, edit and replace the credential information below with your information in the file create_credential.sql: + + * user\_ocid + * tenancy\_ocid + * compartment_ocid + * private\_key - *(Important Note: Put the private key all on a single line.)* + * fingerprint + + ``` + + declare + jo json_object_t; + begin + jo := json_object_t(); + jo.put('user_ocid','ocid1.user.oc1..aabbalbbaa1112233aabb...'); + jo.put('tenancy_ocid','ocid1.tenancy.oc1..aaaaalbbbb1112233aaaab...'); + jo.put('compartment_ocid','ocid1.compartment.oc1..ababalabab1112233ababa...'); + jo.put('private_key','AAAaaaBBB11112222333...AAA111AAABBB222aaa1a...'); + jo.put('fingerprint','01:1a:a1:aa:12:a1:12:1a:ab:12:01:ab:...'); + dbms_vector.create_credential( + credential_name => 'GENAI_CRED', + params => json(jo.to_string)); + end; + / + + ``` +4. Go to /home/oracle/AIdemo +5. sqlplus vector/vector@freepdb1 +6. SQL> @dropcred.sql +7. SQL> @create_cred.sql + diff --git a/langchain-rag-23ai/lab1/images/ignorewarning1.png b/langchain-rag-23ai/lab1/images/ignorewarning1_old.png similarity index 100% rename from langchain-rag-23ai/lab1/images/ignorewarning1.png rename to langchain-rag-23ai/lab1/images/ignorewarning1_old.png diff --git a/langchain-rag-23ai/lab1/images/load_libs.png b/langchain-rag-23ai/lab1/images/load_libs.png new file mode 100644 index 0000000..35f5a73 Binary files /dev/null and b/langchain-rag-23ai/lab1/images/load_libs.png differ diff --git a/langchain-rag-23ai/lab1/lab1.md b/langchain-rag-23ai/lab1/lab1.md index a705916..ce7d631 100644 --- a/langchain-rag-23ai/lab1/lab1.md +++ b/langchain-rag-23ai/lab1/lab1.md @@ -34,19 +34,19 @@ In this lab, you will: 2. From the terminal OS prompt type the following to launch jupyter notebook: -``` - $ cd /home/oracle/AIdemo - $ jupyter notebook -``` + ``` + $ cd /home/oracle/aidemo + $ jupyter notebook + ``` 3. Open the notebook **RAG\_with\_Oracle23ai\_gold1.ipynb**. You can double click or right-click and select **Open**. -![Open RAG notebook](images/openragwithoracle23ai.png) + ![Open RAG notebook](images/openragwithoracle23ai.png) -If you want to enlarge the window and have larger fonts, you can zoom in with the browser. + If you want to enlarge the window and have larger fonts, you can zoom in with the browser. -![Zoom in](images/zoom.png) + ![Zoom in](images/zoom.png) ## Task 2: Run the RAG application code snippets in Jupyter notebook @@ -64,71 +64,81 @@ Let's import the libraries. You may get a warning message as shown below. You can ignore this and click **Run** to proceed. - ![ignore warning](images/ignorewarning1.png) - -```python -# Import libraries and modules - -import sys -import array -import time -import oci -import os -from dotenv import load_dotenv -from PyPDF2 import PdfReader -import oracledb -from oraclevs import OracleVS -from ads.llm import GenerativeAIEmbeddings, GenerativeAI -from sentence_transformers import CrossEncoder -from LangChain.text_splitter import CharacterTextSplitter -from LangChain_community.embeddings import HuggingFaceEmbeddings -from LangChain_community.vectorstores.utils import DistanceStrategy -from LangChain_core.documents import BaseDocumentTransformer, Document -from LangChain_community.llms import OCIGenAI -from LangChain_core.prompts import PromptTemplate -from LangChain.chains import LLMChain -from LangChain_core.runnables import RunnablePassthrough -from LangChain_core.output_parsers import StrOutputParser -print("Successfully imported libraries and modules") -``` + ![Load Libraries](images/load_libs.png) + + ```python + # Import libraries and modules + + import sys + import array + import time + import oci + import os + from dotenv import load_dotenv + from PyPDF2 import PdfReader + #from sentence_transformers import CrossEncoder + from langchain.text_splitter import CharacterTextSplitter + from langchain_community.embeddings import HuggingFaceEmbeddings + from langchain_community.vectorstores.utils import DistanceStrategy + from langchain_community.llms import OCIGenAI + from langchain_core.prompts import PromptTemplate + from langchain.chains import LLMChain + from langchain_core.runnables import RunnablePassthrough + from langchain_core.output_parsers import StrOutputParser + from langchain_community.vectorstores import oraclevs + from langchain_community.vectorstores.oraclevs import OracleVS + from langchain_core.documents import BaseDocumentTransformer, Document + from langchain_community.chat_models.oci_generative_ai import ChatOCIGenAI + from langchain_core.messages import AIMessage, HumanMessage, SystemMessage + import oracledb + + from langchain_openai import ChatOpenAI + from langchain_huggingface import HuggingFaceEmbeddings + + # suppersing warning messages + from tqdm import tqdm, trange + + print("Successfully imported libraries and modules") + + ``` 2. This next code snippet defines the function to include metadata with the chunks. Select the code snippet and click **Run**. -```python -# Function to format and add metadata to Oracle 23ai Vector Store + ```python + # Function to format and add metadata to Oracle 23ai Vector Store -def chunks_to_docs_wrapper(row: dict) -> Document: - """ - Converts text into a Document object suitable for ingestion into Oracle Vector Store. - - row (dict): A dictionary representing a row of data with keys for 'id', 'link', and 'text'. - """ - metadata = {'id': row['id'], 'link': row['link']} - return Document(page_content=row['text'], metadata=metadata) -print("Successfully defined metadata wrapper") -``` + def chunks_to_docs_wrapper(row: dict) -> Document: + """ + Converts text into a Document object suitable for ingestion into Oracle Vector Store. + - row (dict): A dictionary representing a row of data with keys for 'id', 'link', and 'text'. + """ + metadata = {'id': row['id'], 'link': row['link']} + return Document(page_content=row['text'], metadata=metadata) + print("Successfully defined metadata wrapper") + ``` 3. This code loads the environment variables and connects to Oracle Database 23ai with the credentials and connection string. Select the code snippet and click **Run**. -```python -# Load environment variables - -load_dotenv() -username = os.getenv("username") -password = os.getenv("password") -dsn = os.getenv("dsn") -COMPARTMENT_OCID = os.getenv("COMPARTMENT_OCID") -print("The database user name is:",username) -print("Database connection information is:",dsn) - -# Connect to the database - -try: - conn23c = oracledb.connect(user=username, password=password, dsn=dsn) - print("Connection successful!") -except Exception as e: - print("Connection failed!") - -``` + ```python + # Load environment variables + + load_dotenv() + username = os.getenv("username") + password = os.getenv("password") + dsn = os.getenv("dsn") + COMPARTMENT_OCID = os.getenv("COMPARTMENT_OCID") + print("The database user name is:",username) + print("Database connection information is:",dsn) + + # Connect to the database + + try: + conn23c = oracledb.connect(user=username, password=password, dsn=dsn) + print("Connection successful!") + except Exception as e: + print("Connection failed!") + + ``` **The 7 RAG Steps** @@ -138,7 +148,7 @@ The document in our use case is in PDF format. We are loading a PDF document and 4. Select the code snippet and click **Run**. -```python + ```python # RAG Step 1 - Load the document # creating a pdf reader object @@ -149,12 +159,12 @@ The document in our use case is in PDF format. We are loading a PDF document and # print the first page print(pdf.pages[0].extract_text()) -``` + ``` **RAG Step 2 - Transform the document to text** 5. The code transforms each page of the PDF document to text. Click **Run** to execute the code. -```python + ```python # RAG Step 2 - Transform the document to text if pdf is not None: @@ -164,7 +174,7 @@ The document in our use case is in PDF format. We are loading a PDF document and text += page.extract_text() print("You have transformed the PDF document to text format") -``` + ``` **RAG Step 3 - Split the text into chunks** @@ -172,162 +182,156 @@ Our chunk size will be 800 characters, with an overlap of 100 characters with ea 6. Click **Run** to execute the code. -```python - # RAG Step 3 - Chunk the text document into smaller chunks - text_splitter = CharacterTextSplitter(separator="\n",chunk_size=800,chunk_overlap=100,length_function=len) - chunks = text_splitter.split_text(text) - print(chunks[0]) -``` -We print the first chunk for your feedback. + ```python + # RAG Step 3 - Chunk the text document into smaller chunks + text_splitter = CharacterTextSplitter(separator="\n",chunk_size=800,chunk_overlap=100,length_function=len) + chunks = text_splitter.split_text(text) + print(chunks[0]) + ``` + We print the first chunk for your feedback. 7. The code adds metadata such as id to each chunk for the database table. Click **Run** to execute the code. -```python - # Create metadata wrapper to store additional information in the vector store - """ - Converts a row from a DataFrame into a Document object suitable for ingestion into Oracle Vector Store. - - row (dict): A dictionary representing a row of data with keys for 'id', 'link', and 'text'. - """ - docs = [chunks_to_docs_wrapper({'id': page_num, 'link': f'Page {page_num}', 'text': text}) for page_num, text in enumerate(chunks)] - print("Created metadata wrapper with the chunks") + ```python + # Create metadata wrapper to store additional information in the vector store + """ + Converts a row from a DataFrame into a Document object suitable for ingestion into Oracle Vector Store. + - row (dict): A dictionary representing a row of data with keys for 'id', 'link', and 'text'. + """ + docs = [chunks_to_docs_wrapper({'id': page_num, 'link': f'Page {page_num}', 'text': text}) for page_num, text in enumerate(chunks)] + print("Created metadata wrapper with the chunks") -``` -**RAG Step 4 - Set up Oracle AI Vector Search and insert the embedding vectors** + ``` + **RAG Step 4 - Set up Oracle AI Vector Search and insert the embedding vectors** -The embedding model used in this lab is **all-MiniLM-L6-v2** from HuggingFace. **Docs** will point to the text chunks. The connection string to the database is in the object **conn23c**. The table to store the vectors and metadata are in **MY_DEMO4**. We use **DOT_PRODUCT** as the algorithm for the nearest neighbor search. -Note: Embedding models are used to vectorize data. To learn more about embedding models, see the LiveLabs on Oracle AI Vector Search. + The embedding model used in this lab is **all-MiniLM-L6-v2** from HuggingFace. **Docs** will point to the text chunks. The connection string to the database is in the object **conn23c**. The table to store the vectors and metadata are in **MY_DEMO4**. We use **DOT_PRODUCT** as the algorithm for the nearest neighbor search. + Note: Embedding models are used to vectorize data. To learn more about embedding models, see the LiveLabs on Oracle AI Vector Search. 8. Click **Run** to execute the code. -```python - # RAG Step 4 - Using an embedding model, embed the chunks as vectors into Oracle Database 23ai. + ```python + # RAG Step 4 - Using an embedding model, embed the chunks as vectors into Oracle Database 23ai. - # Initialize embedding model - model_4db = HuggingFaceEmbeddings(model_name="sentence-transformers/all-MiniLM-L6-v2") + # Initialize embedding model + model_4db = HuggingFaceEmbeddings(model_name="sentence-transformers/all-MiniLM-L6-v2") - # Configure the vector store with the model, table name, and using the indicated distance strategy for the similarity search and vectorize the chunks - s1time = time.time() - knowledge_base = OracleVS.from_documents(docs, model_4db, client=conn23c, table_name="MY_DEMO4", distance_strategy="DistanceStrategy.DOT_PRODUCT") - #knowledge_base = OracleVS.from_documents(docs, model_4db, client=conn23c, table_name="MY_DEMOOCI", distance_strategy="DistanceStrategy.MAX_INNER_PRODUCT") - s2time = time.time() - print( f"Vectorizing and inserting chunks duration: {round(s2time - s1time, 1)} sec.") + # Configure the vector store with the model, table name, and using the indicated distance strategy for the similarity search and vectorize the chunks + s1time = time.time() + knowledge_base = OracleVS.from_documents(docs, model_4db, client=conn23c, table_name="MY_DEMO4", distance_strategy="DistanceStrategy.DOT_PRODUCT") + #knowledge_base = OracleVS.from_documents(docs, model_4db, client=conn23c, table_name="MY_DEMOOCI", distance_strategy="DistanceStrategy.MAX_INNER_PRODUCT") + s2time = time.time() + print( f"Vectorizing and inserting chunks duration: {round(s2time - s1time, 1)} sec.") -``` + ``` **Have a look into the Oracle Database 23ai to see the table with the vectors and metadata created. It's long, scroll down to see the text chunks.** 9. Take a moment to celebrate. You have successfully uploaded the document, transformed it to text, split into chunks, and embedded its vectors in Oracle Database 23ai -```python + ```python # Take a moment to celebrate. You have successfully uploaded the document, transformed it to text, split into chunks, and embedded its vectors in Oracle Database 23ai print("Yay! You have successfully uploaded the document, transformed it to text, split into chunks, and embedded its vectors in Oracle Database 23ai") -``` + ``` **RAG Step 5 - Build the prompt to query the document** 10. The code issues a prompt related to the document we loaded. Click **Run** to execute the code. -```python - # RAG Step 5 - Build the prompt to query the document - - user_question = ("List the features in Oracle Database 23ai") - print ("The prompt to the LLM will be:",user_question) -``` + ```python + # RAG Step 5 - Build the prompt to query the document + + user_question = ("List 10 new features of 23ai")") + print ("The prompt to the LLM will be:",user_question) + ``` 11. The code records the timing for searching the database. It's quick! Click **Run** to execute the code. -```python -# Setup timings to check performance - - # code not needed, only used for measuring timing - if user_question: - s3time = time.time() - result_chunks=knowledge_base.similarity_search(user_question, 5) - s4time = time.time() - print( f"Search user_question and return chunks duration: {round(s4time - s3time, 1)} sec.") - print("") -``` + ```python + # Setup timings to check performance + + # code not needed, only used for measuring timing + if user_question: + s3time = time.time() + result_chunks=knowledge_base.similarity_search(user_question, 5) + s4time = time.time() + print( f"Search user_question and return chunks duration: {round(s4time - s3time, 1)} sec.") + print("") + ``` **Choose an LLM to generate your response** You have 3 LLM choices in this lab. Choose only one, but you can go back later and choose a different one to see its response. Note: For OpenAI, you **CANNOT** run this in the LiveLabs environment. This sample code is for informational purposes so you can run this example in your own environment. -* Choice 1 - OCI GenAI LLM with meta.llama-2.70b-chat -* Choice 2 - OpenAI ChatGPT 3.5 (CANNOT run in this LiveLabs environment) -* Choice 3 - OCI GenAI LLM with Cohere Command +* Choice 1 - OCI GenAI LLM with meta.llama-3.1-70b-instruct +* Choice 2 - OpenAI gpt-4o-mini (CANNOT run in LiveLabs Sandbox/Green button environment, sample only) + Note: To learn more about using other LLMs and accessing LLMs with secure API keys, see the LiveLabs on LLMs for Oracle AI Vector Search. 12a. **Choice 1** - The code below sets up the OCI GenAI Service LLM to use Meta Llama. Click **Run** to execute the code. ```python - # Choice 1, Set the OCI GenAI LLM - ENDPOINT = "https://inference.generativeai.us-chicago-1.oci.oraclecloud.com" - COMPARTMENT_OCID = COMPARTMENT_OCID - EMBED_MODEL="meta.llama-2-70b-chat" - print(ENDPOINT) - - llmOCI = OCIGenAI( - model_id="meta.llama-2-70b-chat", - service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com", - compartment_id=COMPARTMENT_OCID, - model_kwargs={"temperature": 0.7, "top_p": 0.75, "max_tokens": 2000}, - auth_type="API_KEY", - ) + # Choice 1, Set the OCI GenAI LLM + + ENDPOINT = "https://inference.generativeai.us-chicago-1.oci.oraclecloud.com" + COMPARTMENT_OCID = COMPARTMENT_OCID + #EMBED_MODEL="cohere.command-r-16k" + EMBED_MODEL="meta.llama-3.1-70b-instruct" + print(ENDPOINT) + + # set the LLM to get response + llm = ChatOCIGenAI( + model_id=EMBED_MODEL, + service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com", + compartment_id=COMPARTMENT_OCID, + model_kwargs={"temperature": 0.7, "top_p": 0.75, "max_tokens": 2000}, + auth_type="API_KEY", + ) + print("The LLM model you will use is ",EMBED_MODEL ,"from OCI GenAI Service") + ``` 12b. **Choice 2** - The code below sets up OpenAI LLM. Click **Run** to execute the code. ```python - # Choice 2, Set up OpenAI LLM - - from LangChain.embeddings.openai import OpenAIEmbeddings - from LangChain_community.chat_models import ChatOpenAI - from LangChain.callbacks import get_openai_callback - - # set the LLM to get response - llm = ChatOpenAI( - model_name='gpt-3.5-turbo-16k', - temperature = 0.1, - #openai_api_key = os.getenv("OPENAI_API_KEY"), - openai_api_key="sk-bMV... Add your OpenAI API key here", - max_tokens=2000 - ) - print("The LLM model you will use is from OpenAI") + # Choice 2, Set up OpenAI LLM + + print ("WARNING: The step will fail if the API key is not present or is incorrect.") + print ("Please update the OpenAI_API_key before calling the llm in step 13.") + + from langchain.embeddings.openai import OpenAIEmbeddings + #from langchain_community.chat_models import ChatOpenAI + from langchain_openai import ChatOpenAI + from langchain.callbacks import get_openai_callback + import openai + + # set the LLM to get response + OpenAIllm = ChatOpenAI( + model_name='gpt-4o-mini', + temperature = 0.1, + #openai_api_key = os.getenv("OPENAI_API_KEY"), + #openai_api_key="sk-bMV... Add your OpenAI API key here", + openai_api_key="<==== Replace your key from OpenAI ====>", + max_tokens=2000 + ) + print("The LLM model you will use is OpenAI ChatGPT") ``` -12c. **Choice 3** - The code below sets up OCI GenAI Service to use the Cohere LLM. Click **Run** to execute the code. +13. The code below builds the prompt template to include both the question and the context, and instantiates the knowledge base class to use the retriever to retrieve context from Oracle Database 23ai. Click **Run** to execute the code. -```python - # Choice 3, Set up Cohere LLM - - ENDPOINT = "https://inference.generativeai.us-chicago-1.oci.oraclecloud.com" - COMPARTMENT_OCID = COMPARTMENT_OCID - print(ENDPOINT) - - # set the LLM to get response - llm = OCIGenAI( - model_id="cohere.command", - service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com", - compartment_id=COMPARTMENT_OCID, - model_kwargs={"temperature": 0.7, "top_p": 0.75, "max_tokens": 2000}, - auth_type="API_KEY", - ) - print("The LLM model is cohere.command from OCI GenAI Service") + ```python + # Set up a template for the question and context, and instantiate the database retriever object -``` + template = """Answer the question based only on the following context: + {context} Question: {question} """ + prompt = PromptTemplate.from_template(template) + retriever = knowledge_base.as_retriever(search_kwargs={"k": 10}) + print("The template is:",template) + print(retriever) -13. The code below builds the prompt template to include both the question and the context, and instantiates the knowledge base class to use the retriever to retrieve context from Oracle Database 23ai. Click **Run** to execute the code. - -```python -# Set up the template for the questions and context, and instantiate the database retriever object - template = """Answer the question based only on the following context: - {context} Question: {question} """ - prompt = PromptTemplate.from_template(template) - retriever = knowledge_base.as_retriever() -``` + ``` **RAG Steps 6 and 7 - Invoke the chain** @@ -335,35 +339,45 @@ This is the key part of the RAG application. It is the LangChain pipeline that 14. The code defines the RAG chain process and invokes the chain. Click **Run** to execute the code. -```python - # RAG Steps 6 and 7 Chain the entire process together, retrieve the context, construct the prompt with the question and context, and pass to LLM for the response - - s5time = time.time() - print("We are sending the prompt and RAG context to the LLM, wait a few seconds for the response...") - chain = ( - {"context": retriever, "question": RunnablePassthrough()} - | prompt - | llmOCI - | StrOutputParser() - ) - response = chain.invoke(user_question) - print(user_question) - print(prompt) - print(response) - # Print timings for the RAG execution steps - - s6time = time.time() - print("") - print( f"Send user question and ranked chunks to LLM and get answer duration: {round(s6time - s5time, 1)} sec.") - -``` + ```python + + # RAG Step 6. and 7 - Chain the entire process together, retrieve the context, construct the prompt with the question and context, and pass to LLM for the response + + s5time = time.time() + print("We are sending the prompt and RAG context to the LLM, wait a few seconds for the response...") + chain = ( + {"context": retriever, "question": RunnablePassthrough()} + | prompt + | llm + | StrOutputParser() + ) + + user_question = "list 10 new features of 23ai" + response = chain.invoke(user_question) + print(user_question) + print(prompt) + print(response) + # Print timings for the RAG execution steps + + s6time = time.time() + print("") + print( f"Send user question and ranked chunks to LLM and get answer duration: {round(s6time - s5time, 1)} sec.") + + ``` The code above sets up a processing pipeline where user_question is processed sequentially by retriever, prompt, llmOCI, and StrOutputParser, with each step performing some transformation or analysis on the input. The final result is stored in the variable response. -chain = (...): This line initializes a chain of processing steps. Each step is represented as an object or function. -1. {"context": retriever, "question": RunnablePassthrough()}: This is the first step in the chain. It involves a dictionary with keys "context" and "question" mapped to some objects or functions named retriever and RunnablePassthrough() respectively. -2. | prompt: The | operator is used to chain the output of the previous step with the prompt object or function. This suggests that the output of the first step will be passed as input to prompt. -3. | llmOCI: Similarly, the output of the previous step is passed as input to llmOCI. -4. | StrOutputParser(): Finally, the output of llmOCI is passed through StrOutputParser. -5. response = chain.invoke(user_question): This line invokes the entire chain with the user_question as input and assigns the output to the variable response. + + 1\. *chain = (...):* This line initializes a chain of processing steps. Each step is represented as an object or function. + + 2\. *{"context":* retriever, "question": RunnablePassthrough()}:* This is the first step in the chain. It involves a dictionary with keys "context" and "question" mapped to some objects or functions named retriever and RunnablePassthrough() respectively. + + 3\. *prompt:* The | operator is used to chain the output of the previous step with the prompt object or function. This suggests that the output of the first step will be passed as input to prompt. + + 4\. *llm:* Similarly, the output of the previous step is passed as input to llm. + + 5\. *StrOutputParser():* Finally, the output of llmOCI is passed through StrOutputParser. + + 6\. *response = chain.invoke(user\_question):* This line invokes the entire chain with the user_question as input and assigns the output to the variable response. + **Finish** @@ -387,4 +401,4 @@ You may now [proceed to the next lab](#next). ## Acknowledgements * **Authors** - Vijay Balebail, Milton Wan * **Contributors** - Douglas Hood, Rajeev Rumale -* **Last Updated By/Date** - Milton Wan, May 2024 +* **Last Updated By/Date** - Vijay Balebail, October 2024 diff --git a/langchain-rag-23ai/lab2/lab2.md b/langchain-rag-23ai/lab2/lab2.md index f07e32f..4fdae60 100644 --- a/langchain-rag-23ai/lab2/lab2.md +++ b/langchain-rag-23ai/lab2/lab2.md @@ -18,7 +18,7 @@ In this lab you will run the RAG application interactively using a simple user i 1. From the terminal, go to directory ``` - $ cd /home/oracle/AIdemo + $ cd /home/oracle/aidemo ``` 2. Run the RAG application @@ -51,6 +51,7 @@ In a few seconds the LLM will answer the question and respond with context store - Tell me more about AI Vector Search - Tell me more about new enhancement to SQL - Tell me more about JSON Relational Duality +- List 5 new features of Oracle 23ai database For the Oracle Fiscal Year 2024 Quarter 2 Financial Results document: - Summarize the Oracle 2024 Quarter 2 financial results @@ -62,7 +63,7 @@ Now check out the application code using the UI. From a terminal window type: ``` - $ cd /home/oracle/AIdemo + $ cd /home/oracle/aidemo $ more genai.py ``` @@ -81,4 +82,4 @@ By using AI Vector Search in Oracle Database 23ai, you can build RAG application ## Acknowledgements * **Authors** - Vijay Balebail, Milton Wan * **Contributors** - Rajeev Rumale -* **Last Updated By/Date** - Milton Wan, May 2024 +* **Last Updated By/Date** - Rajeev Rumale, October 2024 diff --git a/langchain-rag-23ai/prepare-setup/prepare-setup.md b/langchain-rag-23ai/prepare-setup/prepare-setup.md new file mode 100644 index 0000000..b5551f1 --- /dev/null +++ b/langchain-rag-23ai/prepare-setup/prepare-setup.md @@ -0,0 +1,51 @@ +# Prepare Setup + +## Introduction +This lab will show you how to download the Oracle Resource Manager (ORM) stack zip file needed to setup the resource needed to run this workshop. This workshop requires a compute instance running the *Oracle Database 23ai on LiveLabs* Marketplace image and a Virtual Cloud Network (VCN). + +*Estimated Lab Time:* 10 minutes + +### Objectives +- Download ORM stack +- Configure an existing Virtual Cloud Network (VCN) + +### Prerequisites +This lab assumes you have: +- An Oracle Free Tier or Paid Cloud account + +## Task 1: Download Oracle Resource Manager (ORM) stack zip file +1. Click on the link below to download the Resource Manager zip file you need to build your environment: [ai-plsql-rag-ll-orm-mkplc-freetier.zip](https://oraclepartnersas.objectstorage.us-ashburn-1.oci.customer-oci.com/p/6Yg5KxmyFSWXouVoBS2yN77St1pCv3AhW7MjW6cetDpssjb37BXFUpqdrKS5ddcN/n/oraclepartnersas/b/aivs-ash/o/ai-plsql-rag-ll-orm-mkplc-freetier.zip) + +2. Save in your downloads folder. + +We strongly recommend using this stack to create a self-contained/dedicated VCN with your instance(s). Skip to *Task 3* to follow our recommendations. If you would rather use an exiting VCN then proceed to the next step as indicated below to update your existing VCN with the required Egress rules. + +## Task 2: Adding Security Rules to an Existing VCN +This workshop requires a certain number of ports to be available, a requirement that can be met by using the default ORM stack execution that creates a dedicated VCN. In order to use an existing VCN the following ports should be added to Egress rules + +| Port |Description | +| :------------- | :------------------------------------ | +| 22 | SSH | +| 6080 | noVNC Remote Desktop | + +1. Go to *Networking >> Virtual Cloud Networks* +2. Choose your network +3. Under Resources, select Security Lists +4. Click on Default Security Lists under the Create Security List button +5. Click Add Ingress Rule button +6. Enter the following: + - Source CIDR: 0.0.0.0/0 + - Destination Port Range: *Refer to above table* +7. Click the Add Ingress Rules button + +## Task 3: Setup Compute +Using the details from the two Tasks above, proceed to the lab *Environment Setup* to setup your workshop environment using Oracle Resource Manager (ORM) and one of the following options: +- Create Stack: *Compute + Networking* +- Create Stack: *Compute only* with an existing VCN where security lists have been updated as per *Task 2* above + +You may now [proceed to the next lab](#next). + +## Acknowledgements +* **Author** - Rene Fontcha, LiveLabs Platform Lead, NA Technology +* **Contributors** - Meghana Banka +* **Last Updated By/Date** - Rajeev Rumale, Database Product Management, August 2024 \ No newline at end of file diff --git a/langchain-rag-23ai/workshops/desktop/index.html b/langchain-rag-23ai/workshops/desktop/index.html new file mode 100644 index 0000000..3b64c4e --- /dev/null +++ b/langchain-rag-23ai/workshops/desktop/index.html @@ -0,0 +1,62 @@ + + + + + + + + + Oracle LiveLabs + + + + + + + + + + + + +
+
+
+
+
+
+
+
+ + + + + diff --git a/langchain-rag-23ai/workshops/desktop/manifest.json b/langchain-rag-23ai/workshops/desktop/manifest.json new file mode 100644 index 0000000..02459ec --- /dev/null +++ b/langchain-rag-23ai/workshops/desktop/manifest.json @@ -0,0 +1,31 @@ +{ + "workshoptitle": "AI Vector Search - 7 Easy Steps to Building a RAG application with Oracle AI Vector Search and LangChain", + "help": "livelabs-help-db_us@oracle.com", + "tutorials": [ + { + "title": "Introduction", + "description": "Introduction", + "filename": "../../introduction/introduction.md" + }, + { + "title": "Lab 0: Set up access to OCI Generative AI Service", + "description": "Configure access to OCI Generative AI Service.", + "filename": "../../lab0/lab0.md" + }, + { + "title": "Lab 1: Build and Run the RAG Application with Oracle AI Vector Search and PLSQL", + "description": "Hands-on lab for building and running the RAG Application.", + "filename": "../../lab1/lab1.md" + }, + { + "title": "Lab 2: Run the RAG application with an interactive UI", + "description": "Run the Streamlit RAG Application.", + "filename": "../../lab2/lab2.md" + }, + { + "title": "Need Help?", + "description": "Solutions to Common Problems and Directions for Receiving Live Help", + "filename": "https://oracle-livelabs.github.io/common/labs/need-help/need-help-freetier.md" + } + ] +} diff --git a/langchain-rag-23ai/workshops/freetier/index.html b/langchain-rag-23ai/workshops/freetier/index.html new file mode 100644 index 0000000..3b64c4e --- /dev/null +++ b/langchain-rag-23ai/workshops/freetier/index.html @@ -0,0 +1,62 @@ + + + + + + + + + Oracle LiveLabs + + + + + + + + + + + + +
+
+
+
+
+
+
+
+ + + + + diff --git a/langchain-rag-23ai/workshops/freetier/manifest.json b/langchain-rag-23ai/workshops/freetier/manifest.json new file mode 100644 index 0000000..8fee04b --- /dev/null +++ b/langchain-rag-23ai/workshops/freetier/manifest.json @@ -0,0 +1,48 @@ +{ + "workshoptitle": "AI Vector Search - 7 Easy Steps to Building a RAG application with Oracle AI Vector Search and LangChain", + "help": "livelabs-help-db_us@oracle.com", + "tutorials": [ + { + "title": "Introduction", + "description": "Introduction", + "filename": "../../introduction/introduction.md" + }, + { + "title": "Get Started", + "description": "This is the prerequisites for customers using own tenancies.", + "filename": "https://oracle-livelabs.github.io/common/labs/cloud-login/cloud-login.md" + }, + { + "title": "Prepare Setup", + "description": "How to download your ORM stack and update security rules for an existing VCN", + "publisheddate": "08/20/2024", + "filename": "../../prepare-setup/prepare-setup.md" + }, + { + "title": "Set up compute instance", + "description": "How to provision the workshop environment and connect to it", + "publisheddate": "06/30/2020", + "filename": "https://oracle-livelabs.github.io/common/labs/setup-compute-generic/setup-compute-novnc.md" + }, + { + "title": "Lab 0: Set up access to OCI Generative AI Service", + "description": "Configure access to OCI Generative AI Service.", + "filename": "../../lab0/lab0.md" + }, + { + "title": "Lab 1: Build and Run the RAG Application with Oracle AI Vector Search and PLSQL", + "description": "Hands-on lab for building and running the RAG Application.", + "filename": "../../lab1/lab1.md" + }, + { + "title": "Lab 2: Run the RAG application with an interactive UI", + "description": "Run the Streamlit RAG Application.", + "filename": "../../lab2/lab2.md" + }, + { + "title": "Need Help?", + "description": "Solutions to Common Problems and Directions for Receiving Live Help", + "filename": "https://oracle-livelabs.github.io/common/labs/need-help/need-help-freetier.md" + } + ] +}