Alter-Ego — Legal RAG & Agentic Assistant

Apprentices navigating the French Code du travail face dense legislative and regulatory articles where version history, cross-references, and external decrees (such as SMIC revaluations) make standard LLM answers prone to hallucination.

How to build a zero-cost, locally executable legal assistant that answers apprenticeship law questions with exact LEGIARTI citations, handles historical article versions and wage calculations deterministically, and strictly abstains or resists prompt injection when out of scope?

Hugging Face AgentPublic/legi dataset (DINUM / DILA LEGI database, Etalab 2.0): 540 in-force (VIGUEUR) Code du travail articles and 1,066 historical article versions (MODIFIE, ABROGE, TRANSFERE) filtered across apprenticeship, young workers, working time, and contract termination rules, plus a 40-question verified golden evaluation set.

Built an ingestion pipeline constructing contextualized chunk_text with legal hierarchy and Nota bene metadata. Benchmarked 5 retrieval modes: BM25Okapi (custom regex preserving article numbers like L6222-35), DINUM baseline, Dense BAAI/bge-m3 (1024-d), Hybrid RRF (k=60), and Hybrid + BAAI/bge-reranker-v2-m3 with exact-number boosting. Paired local Ollama (gemma4:e4b-it-qat / 12b) and cloud API comparison (Gemini 2.5 Flash, GPT-4o, Mistral Small, Claude) with deterministic tools (compute_remuneration for SMIC calculation to ±0.01€, get_article_at_date for temporal queries) and an interactive split-screen web UI.

Hybrid + Rerank achieved 90.3% Recall@5 by article number (76.4% exact version ID, MRR@10 of 0.870), outperforming the DINUM baseline. On the 40-question golden evaluation set, the system achieved 0 errors, 100% accuracy (8/8) on SMIC remuneration calculations and 2024 historical lookups, 100% refusal (4/4) on out-of-scope queries with zero hallucinated citations, and 100% resistance (4/4) against prompt injections at €0.00 local inference cost.

Production-grade RAG and agent engineering blueprint demonstrating retrieval ablation, deterministic tool sandboxing for numerical/temporal precision, and rigorous LLM-as-a-judge evaluation.