Abstract
RAG typically assumes centralized access to documents, which breaks down when knowledge is distributed across private data silos. We propose a secure Federated RAG system built using Flower that performs local silo retrieval, while server-side aggregation and text generation run inside an attested, confidential compute environment, enabling confidential remote LLM inference even in the presence of honest-but-curious or compromised servers. We also propose a cascading inference approach that incorporates a non-confidential third-party model (e.g., Amazon Nova) as auxiliary context without weakening confidentiality.
Abstract (translated)
RAG通常假设文档集中访问，但当知识分布于私有数据孤岛时，该假设便会失效。我们提出一种基于Flower构建的安全联邦RAG系统，该系统在本地数据孤岛执行检索，而服务器端的聚合与文本生成则在经认证的机密计算环境中运行，从而在存在诚实但好奇或被入侵的服务器时，仍能实现机密远程大语言模型推理。我们还提出一种级联推理方法，该方法将非机密第三方模型（如Amazon Nova）作为辅助上下文融入系统，且不会削弱机密性。
URL
https://arxiv.org/abs/2603.25374
