Abstract
There is growing interest in extending average treatment effect (ATE) estimation to incorporate non-tabular data, such as images and text, which may act as sources of confounding. Neglecting these effects risks biased results and flawed scientific conclusions. However, incorporating non-tabular data necessitates sophisticated feature extractors, often in combination with ideas of transfer learning. In this work, we investigate how latent features from pre-trained neural networks can be leveraged to adjust for sources of confounding. We formalize conditions under which these latent features enable valid adjustment and statistical inference in ATE estimation, demonstrating results along the example of double machine learning. We discuss critical challenges inherent to latent feature learning and downstream parameter estimation arising from the high dimensionality and non-identifiability of representations. Common structural assumptions for obtaining fast convergence rates with additive or sparse linear models are shown to be unrealistic for latent features. We argue, however, that neural networks are largely insensitive to these issues. In particular, we show that neural networks can achieve fast convergence rates by adapting to intrinsic notions of sparsity and dimension of the learning problem.
Abstract (translated)
人们对将平均处理效应（ATE）估计扩展到包含非表格数据（如图像和文本），这些数据可能作为混杂因素来源的兴趣日益增加。忽视这些影响可能会导致结果偏倚及科学结论错误。然而，纳入非表格数据需要复杂的特征提取器，并且通常与迁移学习的概念相结合。在本工作中，我们研究了如何利用预训练神经网络的潜在特征来调整混杂因素的影响。我们形式化了这些潜在特征能够使ATE估计中的有效调整和统计推断成立的条件，并通过双重机器学习的例子展示了其结果。我们讨论了源自高维度和表示非唯一性带来的下游参数估计中潜在特征学习所固有的关键挑战。获取具有加性和稀疏线性模型的快速收敛率所需的常见结构假设对于潜在特征来说是不现实的。然而，我们认为神经网络在很大程度上对这些问题不敏感。特别是，我们展示了通过适应于学习问题内在的稀疏度和维度概念，神经网络可以实现快速收敛速度。
URL
https://arxiv.org/abs/2506.14329
