Enterprise Agentic RAG (Scalable Pipeline)

A production-grade, enterprise-level RAG system built with LangGraph, Portkey LLM Gateway, and Gemini Embeddings. The system distinguishes between technical "True Data" and random "Noisy Data" using semantic re-ranking, history-aware planning, and NeMo Guardrails for input/output safety.

Key Features

• Agentic Intelligence: LangGraph for cyclic reasoning, multi-step planning, and conversation memory.
• Guardrails: NeMo Guardrails gate blocks off-topic, jailbreak, and injection inputs before any retrieval.
• LLM Gateway: Portkey routes all LLM calls with automatic fallback between primary and backup Groq keys.
• Enterprise Search: Qdrant Cloud for high-performance vector search + FlashRank for local semantic reranking.
• Gemini Embeddings: Google gemini-embedding-2-preview (3072-dim) via langchain-google-genai.
• Local Document Parsing: PDF, HTML, TXT, DOCX, PPTX parsed entirely on-device — no external OCR service.
• Observability: Full trace nesting with Pydantic Logfire and LangSmith across every agent node.
• Evaluation Suite: RAGAS-powered eval pipeline (6 metrics) with a dedicated Streamlit demo app.

---

Agent Intelligence Flow

graph TD
    User((User)) --> UI[Streamlit UI]
    UI --> API[FastAPI /query]
    API --> Guard{NeMo Guardrails}
    Guard -->|Blocked| UI
    Guard -->|Pass| Planner{Planner Node}
    Planner -->|Conversational| Responder[Responder Node]
    Planner -->|Technical| Retriever[Retriever Node]
    Retriever --> Reranker[FlashRank Local Reranker]
    Reranker --> Responder
    Responder --> UI
    Responder -.-> Memory[(LangGraph MemorySaver)]

Loading

---

Project Structure

├── app/
│   ├── agents/
│   │   └── nodes/       # Planner, Retriever, Responder LangGraph nodes
│   ├── gateway/         # Portkey LLM gateway — primary + fallback Groq routing
│   ├── guardrails/      # NeMo Guardrails input/output filtering
│   ├── ingestion/
│   │   ├── chunking/    # Paragraph-based text splitter (1500 char max)
│   │   └── loaders/     # Local parsers — PDF (pypdf), HTML, TXT, DOCX, PPTX
│   ├── services/
│   │   └── retrieval/   # Gemini embeddings + Qdrant search + FlashRank reranking
│   ├── config.py        # Centralized environment variable management
│   └── main.py          # FastAPI entrypoint — guardrails gate + /query endpoint
├── evals/               # RAGAS evaluation suite + Streamlit 3-tab demo
├── ui/                  # Streamlit chat interface with reasoning step transparency
├── processed_data/      # Auto-generated — parsed & chunked JSON output per document
├── docs/                # Architectural and operational guides (11 docs)
├── DATA/                # Sample datasets (True vs Noisy documentation)
└── requirements.txt     # Pinned dependencies

---

Tech Stack

Layer	Technology
Orchestration	LangChain + LangGraph
LLMs	Groq (Llama 3.3 70B) via Portkey gateway
Guardrails	NeMo Guardrails
Vector DB	Qdrant Cloud
Reranking	FlashRank (local, zero-latency)
Embeddings	Gemini gemini-embedding-2-preview (3072-dim)
Document Parsing	pypdf + pdfplumber (local, no OCR service)
Observability	Pydantic Logfire + LangSmith
Evaluation	RAGAS + custom Tool Correctness (Jaccard)

---

Getting Started

1. Install dependencies

python -m venv tenvv
.\tenvv\Scripts\activate
pip install -r requirements.txt

2. Configure environment

Create a .env file with the following keys:

# Groq Reasoning Engine (Llama 3.3)
GROQ_API_KEY = ""
GROQ_FALLBACK_API_KEY = ""          # second Groq key, or same as primary

# Portkey LLM Gateway
PORTKEY_API_KEY = ""

# Qdrant Vector DB
QDRANT_API_KEY = ""
QDRANT_CLUSTER_ENDPOINT = ""        # e.g. https://your-cluster.cloud.qdrant.io:6333

# Pydantic Logfire Observability
LOGFIRE_TOKEN = ""

# LangSmith
LANGSMITH_TRACING = true
LANGSMITH_ENDPOINT = https://api.smith.langchain.com
LANGSMITH_API_KEY = ""
LANGSMITH_PROJECT = ""

# Streamlit UI → FastAPI
BACKEND_URL = ""                    # e.g. http://localhost:8000

# Eval judge LLM (keep separate from main key to avoid rate-limiting the live app)
JUDGE_GROQ = ""

# Gemini Embeddings
GEMINI_API_KEY = ""

3. Run data ingestion

Parses all documents in DATA/, chunks them, saves metadata to processed_data/, and indexes vectors into Qdrant.

python -m app.ingestion.processor DATA --wipe

Pass --wipe to drop and recreate the Qdrant collection. Omit it to append to an existing collection.

4. Launch the app

# Terminal 1 — FastAPI backend
uvicorn app.main:app --reload --port 8000

# Terminal 2 — Streamlit UI
streamlit run ui/app.py

5. Run the eval suite (optional)

# Requires the FastAPI backend running on :8000
streamlit run evals/app.py

---

Documentation Index

#	Guide	What it covers
01	System Overview	High-level vision and end-to-end flow
02	Ingestion Engine	Document parsing and indexing pipeline
03	Node Intelligence	Planner, Retriever, Responder internals
04	Observability	Logfire + LangSmith tracing
05	Environment Variables	All env vars and configuration reference
06	Known Gotchas	Non-obvious bugs and architectural decisions
07	FlashRank Reranking	Local semantic reranker deep-dive
08	Guardrails	NeMo Guardrails implementation
09	LLM Gateway	Portkey routing, fallback, and observability
10	Evals	RAGAS metrics theory and token budget
11	Evals Pipeline	Live eval pipeline and Streamlit demo

---

Built for High-Scale Enterprise Document Intelligence.