Livoa LogoLivoa


file_loader: Raw PDF/TXT

cleaner: Remove TOC/Footers

chunker: 120-word Overlapping Chunks


prompts: Contextual Prompting

generator: Phi-2 Generation with Retry

qa_parser: Pattern Matching Parsing

evidence: Sentence-level Embedding Match

validators: Reference & Vague Check


dedupe: Semantic Cosine Similarity

interim: Grounded QA Dataset

SimpleRetriever: TF-IDF Vectorization


SFT: Supervised Fine-Tuning

Discriminators: Fact / Style / Safety

RL Loop: PPO / Policy Gradient

Reward Calculation: Fact-Check + Critic Probabilities

evaluation: Final Benchmarked Model

DATA INGESTION & PRE

QA PARSING & VALIDATION

HALLUCINATION REDUCTION - RL

DATASET RETRIEVAL

Diagrams

by Saad

0
0 uses