Production RAG (Without Frameworks)

A minimal, high-performance Retrieval-Augmented Generation (RAG) pipeline built from scratch using raw Python libraries (torch, transformers, faiss) without relying on heavy orchestration frameworks like LangChain or LlamaIndex.

---

What is RAG?

Retrieval-Augmented Generation (RAG) is a technique used to improve the accuracy and reliability of Large Language Models (LLMs) by fetching relevant facts from an external knowledge base (context) before generating a response.

Instead of relying solely on the static training data of the LLM, RAG:

1. Translates a user's query into an embedding vector.
2. Queries a vector database (e.g., FAISS) to retrieve the most semantically relevant text chunks from local documents.
3. Injects those chunks as context into the prompt.
4. Feeds the enriched prompt to the LLM to generate an answer grounded strictly in the provided documents, reducing hallucinations.

---

Architecture Diagram

graph TD
    subgraph Ingestion ["1. Ingestion Pipeline"]
        A[Documents Directory] --> B[Load Documents]
        B --> C[Paragraph-Aware Recursive Chunker]
        C --> D[Embedding Model: BAAI/bge-small-en-v1.5]
        D --> E[FAISS Vector Index]
        E --> F[Save index.faiss & metadata.pkl]
    end

    subgraph Inference ["2. Retrieval & Generation Pipeline"]
        G[User Query] --> H[Embed Query]
        F -.->|Load Vector Index| I[FAISS Semantic Search]
        H --> I
        I --> J[Retrieve Top-K Context Chunks]
        J --> K[Build Prompt with Context]
        G --> K
        K --> L[LLM: Qwen2.5-0.5B-Instruct]
        L --> M[Fact-Grounded Answer]
    end
    
    style Ingestion fill:#f5f7ff,stroke:#5c7cfa,stroke-width:2px;
    style Inference fill:#fff9db,stroke:#fcc419,stroke-width:2px;

Loading

---

Folder Structure

├── app/
│   ├── __init__.py
│   ├── config.py         # Hyperparameters (model names, chunk size, directories)
│   ├── embed.py          # Embedding generation logic using HuggingFace Models
│   ├── generate.py       # LLM generation logic using AutoModelForCausalLM
│   ├── ingest.py         # Document loading, paragraph chunking, and index building
│   ├── prompt.py         # Prompt template formatting and grounding rules
│   ├── retrieve.py       # FAISS index reading and semantic similarity search
│   └── utils.py          # Helper utilities
├── data/
│   ├── documents/        # Folder containing raw source text files (e.g., football.txt)
│   └── index/            # Folder containing generated FAISS index and chunk metadata
├── .gitignore            # Git ignore file (excludes virtual environment and local indices)
├── ingest_data.py        # Executable script to ingest documents and build the database
├── main.py               # Executable entrypoint to run the query retrieval and generation
└── requirements.txt      # Pinpointed Python project dependencies

---

How to Run

1. Prerequisites

Ensure you have Python 3.10+ installed.

2. Set Up Virtual Environment

Create and activate a Python virtual environment:

# Create the environment
python3 -m venv .venv

# Activate it
source .venv/bin/activate

3. Install Dependencies

Install all required libraries listed in requirements.txt:

pip install -r requirements.txt

4. Place Your Documents

Put your .txt files containing knowledge data into the data/documents/ directory. (e.g., data/documents/football.txt).

5. Ingest and Embed Data

Run the ingestion script to chunk your documents, compute embedding vectors using the BAAI/bge-small-en-v1.5 model, and build the FAISS index:

python ingest_data.py

This saves index.faiss and metadata.pkl into the data/index/ folder.

6. Run RAG Query

Execute the main script to query the RAG pipeline:

python main.py

This queries the pipeline with "what is FIFA world cup" and prints the generated answer. To modify the query, edit the query variable in main.py.

---

Example Output

When running python main.py, the system retrieves the most relevant context and generates the following grounded answer:

Answer:

FIFA World Cup is a major international football tournament held annually. It's one of the most popular sporting events globally, attracting millions of viewers worldwide. The tournament typically features high-profile teams from around the world competing against each other in various competitions. The World Cup has been held since 1930, making it one of the longest-running sports tournaments in history. It's known for its intense competition, high stakes, and global significance. The tournament attracts fans from all over the world, who eagerly await the results of each match. Despite its popularity, the World Cup also faces challenges such as limited ticket availability due to expansion plans. In summary, FIFA World Cup is an annual international football tournament featuring top teams from around the globe. It's a significant event in the world of sports and entertainment.