Abstract
We study instrumental variable regression (IVaR) under differential privacy constraints. Classical IVaR methods (like two-stage least squares regression) rely on solving moment equations that directly use sensitive covariates and instruments, creating significant risks of privacy leakage and posing challenges in designing algorithms that are both statistically efficient and differentially private. We propose a noisy two-state gradient descent algorithm that ensures $\rho$-zero-concentrated differential privacy by injecting carefully calibrated noise into the gradient updates. Our analysis establishes finite-sample convergence rates for the proposed method, showing that the algorithm achieves consistency while preserving privacy. In particular, we derive precise bounds quantifying the trade-off among privacy parameters, sample size, and iteration-complexity. To the best of our knowledge, this is the first work to provide both privacy guarantees and provable convergence rates for instrumental variable regression in linear models. We further validate our theoretical findings with experiments on both synthetic and real datasets, demonstrating that our method offers practical accuracy-privacy trade-offs.
Abstract (translated)
我们研究了在差分隐私约束下的工具变量回归（IVaR）。传统的IVaR方法（如两阶段最小二乘法回归）依赖于直接使用敏感协变量和工具变量求解矩方程，这带来了显著的隐私泄露风险，并且在设计既统计有效又满足差分隐私算法方面存在挑战。我们提出了一种带有噪声的两阶段梯度下降算法，通过向梯度更新中注入精心校准的噪音来确保$\rho$-零集中差分隐私。我们的分析建立了所提方法的有限样本收敛率，表明该算法在保持隐私的同时实现了渐近一致性。特别地，我们推导了精确界来量化隐私参数、样本量和迭代复杂性之间的权衡关系。据我们所知，这是首次为线性模型中的工具变量回归同时提供隐私保证和可证明的收敛速率的研究工作。我们进一步通过在合成数据集和真实数据集上的实验验证了理论发现，并表明我们的方法提供了实用的精度-隐私权衡。
URL
https://arxiv.org/abs/2509.22794
