> ## Documentation Index
> Fetch the complete documentation index at: https://genai.codewithsiva.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Introduction to RAG

> Understand the core concepts, architecture, and benefits of Retrieval-Augmented Generation

Large Language Models (LLMs) are incredibly powerful, but they have two core limitations: they suffer from **hallucinations** (fabricating facts convincingly) and their knowledge is static (limited to their pre-training training cut-off date). **Retrieval-Augmented Generation (RAG)** solves these problems by grounding model answers in private, external data.

### 1. What is RAG & Why is it Needed?

Instead of relying solely on the LLM's internal weights to generate answers, RAG queries an external datasource to retrieve relevant documents matching the user's question, and then passes those documents to the LLM as context.

#### Why not just fine-tune?

* **Real-time Updates**: RAG can access live data (like database records or APIs) instantly. Fine-tuning is static and slow.
* **Cost & Time**: Fine-tuning requires renting GPUs and running training runs. RAG connects to databases dynamically at runtime.
* **Access Control**: RAG lets you filter documents based on user roles (e.g. Employee A cannot query Employee B's salary documents). Fine-tuned weights expose all information to all users.
* **Verifiability**: RAG outputs can cite source documents (e.g., *"According to Page 12 of the HR Manual..."*), whereas fine-tuned outputs cannot be traced.

### 2. The RAG Pipeline Architecture

RAG systems split their operations into two distinct phases: a one-time/offline **Ingestion Phase** to prepare the database, and an online **Retrieval & Generation Phase** triggered by user queries.

#### Phase 1: Ingestion (Offline/One-time Process)

Before the system can answer questions, raw documents must be processed and indexed:

```text theme={null}
┌───────────┐     ┌────────┐     ┌───────────┐     ┌───────────┐     ┌───────────┐
│ Raw Docs  │ ──> │ Loader │ ──> │ Splitter  │ ──> │ Embedding │ ──> │ Vector DB │
│ (PDF/TXT) │     │ (Text) │     │ (Chunks)  │     │ (Vectors) │     │ (Storage) │
└───────────┘     └────────┘     └───────────┘     └───────────┘     └───────────┘
```

1. **Loading**: Read raw file formats (PDFs, Markdown, CSVs) and convert them to LangChain `Document` objects.
2. **Chunking**: Split large documents into smaller, semantically cohesive chunks.
3. **Embedding**: Convert text chunks into numerical vectors (embeddings) using an embedding model.
4. **Storage**: Save vectors and metadata in a Vector Database for fast retrieval.

#### Phase 2: Retrieval & Generation (Online/Runtime Process)

When a user asks a question, the application executes the following runtime flow:

```text theme={null}
                  ┌──────────────────────┐
                  │      User Query      │
                  └──────────┬───────────┘
                             │
            ┌────────────────┴────────────────┐
            ▼                                 ▼
     ┌─────────────┐                   ┌──────────────┐
     │  Vector DB  │                   │              │
     │  (Search)   │                   │              │
     └──────┬──────┘                   │              │
            │                          │    Prompt    │
            ▼                          │   Template   │
     ┌─────────────┐                   │              │
     │  Retrieved  │                   │              │
     │   Chunks    │ ────────────────> │              │
     └─────────────┘                   └──────┬───────┘
                                              │
                                              ▼
                                       ┌──────────────┐
                                       │     LLM      │ ──> [Answer]
                                       │  (Generate)  │
                                       └──────────────┘
```

1. **Retrieval**: The user's query is converted to a vector embedding and used to query the Vector DB, which returns the top-K most similar text chunks.
2. **Augmentation**: Both the original user query and the retrieved context chunks are inserted into the prompt template.
3. **Generation**: The compiled prompt is sent to the LLM, which synthesizes a grounded response.

### 3. Lexical Search vs. Semantic Search

RAG systems rely on **Semantic Search** (dense retrieval) rather than traditional keyword-matching search:

* **Traditional (Lexical) Search**: Looks for exact word matches (e.g. TF-IDF or BM25). If you search for *"automobile repair"*, it will miss documents containing *"car mechanics"* because the exact characters do not match.
* **Semantic Search**: Converts text into vector embeddings representing meaning. It knows that *"automobile"* and `"car"` are conceptually close, returning relevant matches even without direct keyword intersections.

### 4. Practice Exercises

#### Practice 1: Hallucination Mitigation

Explain how RAG reduces the occurrence of model hallucinations compared to open-ended generation.

<Accordion title="Solution">
  * **Open-ended Generation**: The model relies on predicting the next token based purely on its training parameters. If it doesn't know a fact, it continues predicting the most statistically probable next words, generating incorrect facts (hallucinations).
  * **RAG**: The model is restricted to a prompt template instruction (e.g., *"Answer the query ONLY using the provided context."*). Since the facts are supplied directly in the prompt, the model acts as a summarization/synthesizer, drastically reducing fabricated claims.
</Accordion>

### Summary

* **RAG Benefits**: Solves LLM hallucinations and static training constraints by dynamically grounding answers in private, external data sources.
* **Pipeline Architecture**: Segregated into a one-time/offline **Ingestion Phase** (Loading $\rightarrow$ Chunking $\rightarrow$ Embedding $\rightarrow$ Vector Storage) and an online **Retrieval & Generation Phase** (querying the database, injecting context into the prompt, and generating the response).
* **Search Mechanics**: Utilizes **Semantic Search** (dense vector comparison) instead of keyword exact matches (lexical search) to retrieve relevant records based on meaning.