Multimodal Retrieval-Augmented Generation (RAG) System for NTRO Intelligence Enhancement

Offline & Secure Operation (NTRO Mandate)

Data Sources (Input)

📄 Classified Documents (PDF, DOCX)

📡 Satellite & Surveillance Images

🎙️ Audio Intelligence Recordings

Processing Engine (FastAPI Backend)

Text Extraction (PyMuPDF, docx)

Image Captioning/Detection (BLIP/Florence-2 + YOLO)

Audio Transcription/ID (Whisper + pyannote.audio)

Embedding & Retrieval

Multimodal Embeddings (Sentence Transformers + CLIP)

💾 ChromaDB Vector Store (Semantic Storage)

RAG Generation

User Query ➡️

Retrieval & Context Selection

🧠 Local LLM (Phi-3-mini) Secure Response Generation

NTRO Intelligence Insights (Output)

Situational Awareness Reports

Technical Intelligence Extraction

⬆️ Faster Decision-Making

rag flowchart