Abstract
Accurately tracking food consumption is crucial for nutrition and health monitoring. Traditional approaches typically require specific camera angles, non-occluded images, or rely on gesture recognition to estimate intake, making assumptions about bite size rather than directly measuring food volume. We propose the FoodTrack framework for tracking and measuring the volume of hand-held food items using egocentric video which is robust to hand occlusions and flexible with varying camera and object poses. FoodTrack estimates food volume directly, without relying on intake gestures or fixed assumptions about bite size, offering a more accurate and adaptable solution for tracking food consumption. We achieve absolute percentage loss of approximately 7.01% on a handheld food object, improving upon a previous approach that achieved a 16.40% mean absolute percentage error in its best case, under less flexible conditions.
Abstract (translated)
准确追踪食物摄入量对于营养和健康监测至关重要。传统方法通常需要特定的拍摄角度、非遮挡图像或依赖手势识别来估计摄取量，它们会假设咬食大小而非直接测量食物体积。我们提出了一种名为FoodTrack的框架，用于使用第一人称视角视频（egocentric video）追踪并测量手持食物物品的体积，该方法能够应对手部遮挡问题，并且对不同相机和对象姿态变化具有灵活性。FoodTrack直接估计食物体积，而不依赖于摄取手势或固定咬食大小假设，提供了一种更准确、更具适应性的解决方案来追踪食物摄入量。我们在手持食品物体上实现了约7.01%的绝对百分比损失，在之前一种方法的最佳情况下也仅达成了16.40%的平均绝对百分比误差（在更为严格的条件下）。 该框架具有更高的准确性，并且能够在更加复杂的使用环境中有效运作，从而为食物摄入量的精确追踪和监测提供了新的可能。
URL
https://arxiv.org/abs/2505.04055
