GPT models demonstrated remarkable capabilities in understanding and generating text. However, they sometimes struggle to provide accurate responses and may even offer factually incorrect answers. These models are probabilistic by nature, meaning they generate the most likely response, which is not necessarily the most accurate. In other words, GPT models will always provide an answer, even if it is incorrect. These models are also constrained by their initial training set and may produce incorrect responses when tasked with generating context-specific text that was not included in their initial training data.
GPT 模型在理解與產生文字方面展現出卓越的能力。然而，它們有時難以提供準確的回應，甚至可能給出事實錯誤的答案。這些模型本質上是機率性的，意即它們會產生最有可能的回應，但這不一定是最準確的回應。換句話說，GPT 模型總是會給出一個答案，即使該答案是錯誤的。這些模型也受限於其初始訓練資料集，當被要求產生未包含在初始訓練資料中的特定情境文字時，可能產生不正確的回應。
There are several ways that you can add topic-specific knowledge to a GPT model. The most effective, but also the most time-consuming and expensive is training a GPT model from scratch. To properly train such a model, you would need enormous amounts of training data and a very large and performant GPU cluster. For example, ChatGPT-3 training was estimated to require around 1000 high-end GPUs and about 34 days of training, costing around $4M.
有幾種方法可以為 GPT 模型加入特定主題的知識。最有效但也最耗時、最昂貴的方法，是從頭訓練一個 GPT 模型。要正確訓練這樣的模型，你需要大量的訓練資料，以及效能非常強大的大型 GPU 叢集。舉例來說，據估計 ChatGPT-3 的訓練需要約 1000 顆高階 GPU，以及約 34 天的訓練時間，成本約為 400 萬美元。
A much more cost-effective method for enhancing the knowledge of GPT models involves fine-tuning. In this approach, you take a pre-trained GPT model and train it using a smaller dataset that is tailored to your specific needs. For example, you could reformat the entire CCNA course to serve as a training dataset. During fine-tuning, only a small subset of the model's parameters, such as the weights and biases of the neural networks, are modified. However, altering the parameters of a pre-trained model carries risks, as the model can forget previously learned information, a phenomenon known as "catastrophic forgetting." Although less expensive than training from scratch, fine-tuning still demands considerable expertise to prevent catastrophic forgetting.
一種更具成本效益的方法是透過微調（fine-tuning）來增強 GPT 模型的知識。在這種做法中，你會取用一個已預先訓練好的 GPT 模型，並使用一個針對你特定需求量身打造的較小資料集來進行訓練。舉例來說，你可以將整個 CCNA 課程重新格式化，作為訓練資料集。在微調過程中，只有模型參數的一小部分（例如神經網路的權重與偏差）會被修改。然而，變更預先訓練模型的參數存在風險，因為模型可能會忘記先前學到的資訊，這種現象稱為「災難性遺忘」（catastrophic forgetting）。雖然微調比從頭訓練便宜，但仍需要相當豐富的專業知識，以避免發生災難性遺忘。
Both training and fine-tuning are unsuitable for LLMs that require frequent updates with the latest information, due to the inherently high demands of both processes.
訓練與微調都不適用於需要頻繁更新最新資訊的 LLM，因為這兩種流程本身的需求都很高。
Finally, RAG was designed as a cost-effective and risk-free method to provide topic-specific context to a query, vastly improving accuracy of the responses that are grounded in factual data. This method is based on external datasources, used as a knowledge base, and does not change any of the model parameters. The low cost and the ability to simply add or update relevant information made RAG very popular in the AI community. In this topic you will learn how a RAG system works and discover additional ML concepts, such as tokenization, chunking, embedding, and semantic search.
最後，RAG 被設計成一種具成本效益且無風險的方法，能為查詢提供特定主題的情境，大幅提升以事實資料為依據之回應的準確性。這種方法以外部資料來源作為知識庫，並不會變更任何模型參數。低成本以及能輕易新增或更新相關資訊的特性，使得 RAG 在 AI 社群中廣受歡迎。在本主題中，你將學到 RAG 系統的運作方式，並了解更多 ML 概念，例如詞元化（tokenization）、分塊（chunking）、嵌入（embedding）與語意搜尋（semantic search）。
Overview of RAG
RAG 概述
RAG integrates the content of the files that you provided with the query—prompt.
RAG 將你所提供檔案的內容與查詢（提示）加以整合。
The content from the files provides real, factual information, which is processed together with the query by the Large Language Model (LLM), as shown in the figure.
如圖所示，來自檔案的內容提供了真實、以事實為依據的資訊，並與查詢一起由大型語言模型（LLM）處理。
This is very similar as writing the required context into a prompt together with a query. For example, you could paste the output of a "show running-config" command into the prompt and include a query, such as: "Is SSH enabled?" The LLM will find the relevant information in the prompt itself and provide an accurate answer. You can only imagine what would happen if you forgot to paste the output of the command. The described approach is good enough for shorter contexts and simple queries. For longer contexts and complicated queries, the context-in-prompt method becomes increasingly impractical and even impossible to implement due to the length limitations of the prompt itself.
這與將所需情境連同查詢一起寫入提示中的做法非常類似。舉例來說，你可以將「show running-config」指令的輸出貼到提示中，並加入一個查詢，例如：「是否已啟用 SSH？」LLM 會在提示本身中找到相關資訊，並提供準確的答案。你可以想像，如果你忘了貼上該指令的輸出會發生什麼事。這種做法對於較短的情境與簡單的查詢來說已經足夠。但對於較長的情境與複雜的查詢，由於提示本身的長度限制，「將情境放入提示」的做法會變得愈來愈不切實際，甚至無法實作。
RAG uses an external database to store the contexts and retrieves relevant information from the database on the fly when a query is passed into the system. In our example, the output of the "show running-config" command would first be stored into the database. When you prompt the RAG system, it uses the query to find relevant information in the database, in this case, it would look for anything Secure Socket Shell (SSH) related and extract this information from the database. Next, the retrieved configuration regarding SSH and the prompt are sent to the LLM for processing, which provides an accurate response. Let's explore how this entire process works in a bit more detail.
RAG 使用外部資料庫來儲存情境內容，並在查詢傳入系統時即時從資料庫中檢索相關資訊。在我們的範例中，「show running-config」指令的輸出會先被儲存到資料庫中。當你向 RAG 系統提出提示時，系統會使用該查詢在資料庫中尋找相關資訊，在此例中，它會尋找任何與安全殼層（SSH）相關的內容，並從資料庫中擷取這些資訊。接著，檢索到的 SSH 相關設定與提示會一併送交 LLM 處理，進而提供準確的回應。讓我們更詳細地探討整個流程的運作方式。
How RAG Works?
RAG 如何運作？
RAG is a circular process, as shown in the following figure.
如下圖所示，RAG 是一個循環流程。
The entire RAG process can be generalized into several steps:
整個 RAG 流程可概略分為幾個步驟：
- Preprocessing: The content that is used for context can come from various sources, such as ordinary files, databases, or even live feeds. The RAG system extracts the text content from the various sources and splits it into smaller, more manageable pieces called tokens. A token can be a word, a phrase, or even only a part of a word, depending on the tokenizer used. The tokens are then grouped into meaningful chunks, such as noun or verb phrases. For example, the line "transport input ssh" from the output of the "show running-config" command might be split into the following tokens: ["transport", "in", "put", "SSH"]. These tokens might be grouped into the following chunks: ["transport input", "SSH"]. The grouping into chunks is especially useful in tasks that benefit from understanding higher-order structures within the text. The splitting of text is an extremely important part of the RAG process, since it significantly impacts the quality and efficiency of context retrieval. The process ensures that the RAG system can handle and process large quantities of text efficiently.預先處理：用作情境的內容可以來自各種來源，例如一般檔案、資料庫，甚至即時串流資料。RAG 系統會從這些不同來源擷取文字內容，並將其拆分成更小、更易於管理的片段，稱為詞元（token）。根據所使用的詞元化工具（tokenizer）不同，一個詞元可以是一個字、一個片語，甚至只是單字的一部分。接著，這些詞元會被歸類成有意義的區塊（chunk），例如名詞片語或動詞片語。舉例來說，「show running-config」指令輸出中的「transport input ssh」這一行，可能會被拆分成以下詞元：[「transport」、「in」、「put」、「SSH」]。這些詞元可能會被歸類成以下區塊：[「transport input」、「SSH」]。將內容歸類成區塊，對於需要理解文字中高階結構的任務特別有用。文字的拆分是 RAG 流程中極為重要的一環，因為它會大幅影響情境檢索的品質與效率。這個流程確保 RAG 系統能有效處理與運用大量文字。
- Embedding: Once the text is processed, or consumed in the jargon of RAG, it has to be transformed in a format optimized for search and retrieval—a process called embedding. During this phase, the chunks are passed to a specialized LLM called an embedding model, which converts them into multidimensional vectors (arrays of numbers). These vectors function similarly to geographical coordinates on a map, with each point symbolizing the underlying meaning of the text. Essentially, the numerical values within these vectors capture the essence of what the text conveys. Just as points that are close together on a map represent locations that are near each other, vectors that are close in distance indicate texts that share similar meanings or themes. The vectors are then stored in a specialized vector database, which functions like a dictionary where the lookup key is the vector, and the associated value is the original text. This step ensures that the semantic meaning of the text is preserved and can be efficiently accessed.嵌入：文字經過處理（在 RAG 術語中稱為消耗）後，必須轉換成最適合搜尋與檢索的格式，這個過程稱為嵌入。在此階段，這些區塊會被送入一個稱為嵌入模型的專用 LLM，將它們轉換成多維向量（數字陣列）。這些向量的運作方式類似地圖上的地理座標，每個點代表該文字所隱含的意義。基本上，這些向量中的數值擷取了文字所傳達內容的本質。就如同地圖上距離相近的點代表彼此相近的地點，距離相近的向量則代表意義或主題相似的文字。這些向量接著會被儲存在一個專用的向量資料庫中，其運作方式就像一本字典，查詢鍵是向量，對應的值則是原始文字。這個步驟確保文字的語意能被保留，並能被有效存取。
- Retrieval: When you enter a query into the RAG system, it is also converted into vectors by the same process as the text from the files. The system calculates the "distances" between these vectors and the stored vectors to quickly determine which texts are closest to the query, indicating their relevance. The system then gathers the most relevant texts based on the closest vectors, using them as context for response generation. This method, which is known as semantic search, ensures that the most contextually relevant information is retrieved. For example, the query "Is SSH enabled?" is semantically similar to "ip ssh version 2" and "transport input ssh" lines from the "show running-config" command output due to the presence of the word SSH. The two lines would be collected as context for the example query.檢索：當你在 RAG 系統中輸入查詢時，該查詢也會透過與檔案文字相同的流程被轉換成向量。系統會計算這些向量與已儲存向量之間的「距離」，以快速判定哪些文字與查詢最接近，藉此表示其相關程度。系統接着會根據距離最近的向量收集最相關的文字，作為產生回應的情境。這種方法稱為語意搜尋，能確保檢索到與情境最相關的資訊。舉例來說，查詢「是否已啟用 SSH？」由於包含 SSH 這個字，會與「show running-config」指令輸出中的「ip ssh version 2」及「transport input ssh」這兩行在語意上相似。這兩行內容就會被收集起來，作為此範例查詢的情境。
- Generation:Both the prompt and the context are passed to the general LLM in their original text format, allowing the LLM to produce a factually correct and contextually accurate response.生成：提示與情境都會以原始文字格式一起送交一般 LLM，讓 LLM 能產生事實正確且情境準確的回應。
RAG in Network Operations
網路維運中的 RAG
One of the primary applications of RAG in network operations is the automation of troubleshooting and diagnostics. When a network issue arises, a RAG system can quickly analyze the query describing the problem and retrieve relevant historical data, previous incident reports, and documented solutions from a vast database. By accessing this collective knowledge, the system can generate insightful, contextually relevant suggestions or steps to resolve the issue, significantly speeding up the troubleshooting process.
RAG 在網路維運中的主要應用之一，是疑難排解與診斷的自動化。當網路問題發生時，RAG 系統能快速分析描述問題的查詢，並從龐大的資料庫中檢索相關的歷史資料、過往事件報告，以及已記錄的解決方案。透過存取這些集體知識，系統能產生具有洞察力、與情境相關的建議或解決步驟，大幅加快疑難排解的速度。
Another potential use is in predictive maintenance. RAG systems can process descriptions or logs of network behavior to forecast potential failures or identify parts of the network that might require attention. By combining past data with real-time inputs, these systems can suggest preemptive actions that prevent downtime.
另一項潛在用途是預測性維護。RAG 系統能處理有關網路行為的描述或記錄，以預測潛在故障，或找出網路中可能需要關注的部分。透過結合過去資料與即時輸入，這些系統能建議預防性措施，以避免停機。
RAG can handle user queries in real time, providing technical support, or guiding network configuration changes. For instance, if a network engineer asks how to configure a specific device or troubleshoot an error, the RAG system can generate precise, step-by-step configuration instructions or troubleshooting guides by retrieving similar cases and their resolutions.
RAG 能即時處理使用者查詢，提供技術支援，或引導網路設定變更。舉例來說，若網路工程師詢問如何設定特定裝置或排除某個錯誤，RAG 系統能藉由檢索類似案例及其解決方式，產生精確、逐步的設定說明或疑難排解指南。
RAG can also facilitate the documentation process. As network configurations change or new issues are resolved, the system can automatically generate updated documentation that accurately reflects the latest network state and troubleshooting procedures. This not only saves time but also ensures that the documentation is comprehensive and up-to-date.
RAG 也能促進文件撰寫流程。隨著網路設定變更或新問題獲得解決，系統能自動產生更新後的文件，準確反映最新的網路狀態與疑難排解程序。這不僅能節省時間，也能確保文件內容完整且與時俱進。