Abstract
Retrieval-Augmented Generation (RAG) has emerged as a powerful approach for grounding Large Language Model (LLM)-based chatbot responses on external knowledge. However, existing RAG studies typically assume well-structured textual sources (e.g. Wikipedia or curated datasets) and perform retrieval and generation at query time, which can limit their applicability in real-world chatbot scenarios. In this paper, we present HybridRAG, a novel and practical RAG framework towards more accurate and faster chatbot responses. First, HybridRAG ingests raw, unstructured PDF documents containing complex layouts (text, tables, figures) via Optical Character Recognition (OCR) and layout analysis, and convert them into hierarchical text chunks. Then, it pre-generates a plausible question-answer (QA) knowledge base from the organized chunks using an LLM. At query time, user questions are matched against this QA bank to retrieve immediate answers when possible, and only if no suitable QA match is found does our framework fall back to an on-the-fly response generation. Experiments on OHRBench demonstrate that our HybridRAG provides higher answer quality and lower latency compared to a standard RAG baseline. We believe that HybridRAG could be a practical solution for real-world chatbot applications that must handle large volumes of unstructured documents and lots of users under limited computational resources.
Abstract (translated)
基于检索的增强生成（Retrieval-Augmented Generation，简称RAG）已成为将大型语言模型（Large Language Model, LLM）驱动的聊天机器人响应与外部知识相结合的一种强大方法。然而，现有的RAG研究通常假设存在结构良好的文本来源（例如维基百科或经过整理的数据集），并且在查询时执行检索和生成操作，这可能限制了它们在现实世界中的聊天机器人场景中的适用性。本文中，我们介绍了HybridRAG，这是一种新的、实用的RAG框架，旨在实现更准确且更快捷的聊天机器人响应。 首先，HybridRAG通过光学字符识别（OCR）和布局分析摄取包含复杂布局（如文本、表格和图片）的原始未结构化的PDF文档，并将它们转换为层次化的文本块。然后，使用一个大型语言模型从组织好的这些块中预先生成一个可能的问题-答案（QA）知识库。在查询时，用户提出的问题与这个QA数据库进行匹配，以检索即时回答；如果找不到合适的QA匹配，则我们的框架会回退到实时响应生成。 在OHRBench上的实验表明，HybridRAG相比标准的RAG基线提供了更高的答案质量和更低的延迟。我们认为HybridRAG可能是处理大量未结构化文档和众多用户（且计算资源有限）的真实世界聊天机器人应用的一种实用解决方案。
URL
https://arxiv.org/abs/2602.11156
