An Empirical Study on the Impact of Evaluator-Optimizer Retrieval-Augmented Generation (RAG) Framework on Embedding Sensitivity
Publication Date
1-1-2026
Document Type
Conference Proceeding
Publication Title
Proceedings of the 2026 IEEE International Conference on Industry 4 0 Artificial Intelligence and Communications Technology Iaict 2026
DOI
10.1109/IAICT71158.2026.11620982
First Page
475
Last Page
481
Abstract
Language based video reasoning (LBVR) systems rely on Retrieval-Augmented Generation (RAG) over transcripts and captions of videos. However, naive single-pass RAG pipelines are highly sensitive to embedding and retrieval choices, hence they are prone to hallucination especially when dealing with technical and reasoning corpus. This constrains developers to utilize computationally expensive embedding models and retrieval strategies. This work investigates an Evaluator-Optimizer RAG framework specifically designed for academic reasoning and research focused LBVR. The framework audits and improves a first pass answer by validating logical consistency, grounding, and relevance. We implement a cross-reflection approach to avoid confirmation bias by using GPT-4o for first pass generation and Gemini-3-Pro for feedback refinement. We compare this framework with baseline implementations integrated with state-of-the-art (SOTA) embedding model (BGE). This study demonstrates that feedback-based cross-reflection with lightweight embedding model (MiniLM) performs on par with baselines, enabling implementation of research grade LBVR systems in low compute environments. For evaluation we utilized RAGAS framework rather than static n-grams metrics to retain the semantic meaning of the responses and discussing the trade-offs between faithfulness and answer accuracy. Quantitative evaluations demonstrate that this lightweight configuration outperforms baseline implementations by +0.075 on average, yielding highly reliable responses with an average Faithfulness score of 0.913 and a Response Groundedness score of 0.987. Our results indicate that cross-reflective feedback driven LBVR RAG systems for academia can reduce the dependence on compute resources opening opportunities for deployment in resource-constraint settings such as edge devices.
Keywords
Agentic RAG, Evaluator-Optimizer Framework, Gemini 3 Pro, Large Language Models (LLM), RAGAS, Retrieval-Augmented Generation (RAG), Video Question Answering (Video QA)
Department
Computer Science
Recommended Citation
Udayan Atreya, Mohammad Adil Ansari, and Navrati Saxena. "An Empirical Study on the Impact of Evaluator-Optimizer Retrieval-Augmented Generation (RAG) Framework on Embedding Sensitivity" Proceedings of the 2026 IEEE International Conference on Industry 4 0 Artificial Intelligence and Communications Technology Iaict 2026 (2026): 475-481. https://doi.org/10.1109/IAICT71158.2026.11620982
