Abstract
Light Detection and Ranging (LiDAR) technology in consumer-grade mobile devices can be used as a replacement for traditional background removal and compositing techniques. Unlike approaches such as chroma keying and trained AI models, LiDAR's depth information is independent of subject lighting, and performs equally well in low-light and well-lit environments. We integrate the LiDAR and color cameras on the iPhone 15 Pro Max with GPU-based image processing. We use Apple's SwiftUI and Swift frameworks for user interface and backend development, and Metal Shader Language (MSL) for realtime image enhancement at the standard iPhone streaming frame rate of 60 frames per second. The only meaningful limitations of the technology are the streaming bandwidth of the depth data, which currently reduces the depth map resolution to 320x240, and any pre-existing limitations of the LiDAR IR laser to reflect accurate depth from some materials. If the LiDAR resolution on a mobile device like the iPhone can be improved to match the color image resolution, LiDAR could feasibly become the preeminent method of background removal for video applications and photography.
Abstract (translated)
光探测与测距（LiDAR）技术在消费级移动设备中的应用可以作为传统背景移除和合成技术的替代方案。与传统的色键技术和训练有素的人工智能模型相比，LiDAR提供的深度信息不受物体照明条件的影响，在低光和强光环境下表现同样出色。我们将在iPhone 15 Pro Max上集成LiDAR和彩色相机，并结合基于GPU的图像处理技术进行开发。 我们使用Apple的SwiftUI和Swift框架来构建用户界面和后端功能，同时采用Metal Shader Language（MSL）实现实时图像增强，以达到标准的iPhone流媒体帧率60帧每秒。目前这项技术的主要限制在于深度数据的传输带宽，这导致当前的深度图分辨率只能达到320x240像素，并且某些材料对LiDAR红外激光反射不准确的问题也会影响性能。 如果未来移动设备如iPhone上的LiDAR分辨率能够提升至与彩色图像相同的分辨率水平，那么LiDAR技术有望成为视频应用和摄影中背景移除的首选方法。
URL
https://arxiv.org/abs/2508.20250
