Abstract
Models for monocular shape reconstruction of surfaces with diffuse reflection -- shape from shading -- ought to produce distributions of outputs, because there are fundamental mathematical ambiguities of both continuous (e.g., bas-relief) and discrete (e.g., convex/concave) varieties which are also experienced by humans. Yet, the outputs of current models are limited to point estimates or tight distributions around single modes, which prevent them from capturing these effects. We introduce a model that reconstructs a multimodal distribution of shapes from a single shading image, which aligns with the human experience of multistable perception. We train a small denoising diffusion process to generate surface normal fields from $16\times 16$ patches of synthetic images of everyday 3D objects. We deploy this model patch-wise at multiple scales, with guidance from inter-patch shape consistency constraints. Despite its relatively small parameter count and predominantly bottom-up structure, we show that multistable shape explanations emerge from this model for ''ambiguous'' test images that humans experience as being multistable. At the same time, the model produces veridical shape estimates for object-like images that include distinctive occluding contours and appear less ambiguous. This may inspire new architectures for stochastic 3D shape perception that are more efficient and better aligned with human experience.
Abstract (translated)
模型应为单目表面形状复原--来自阴影的形状，应该产生输出分布，因为连续（例如，浮雕）和离散（例如，凸/凹）数学不确定性也在人类经验中存在。然而，当前模型的输出仅限于单点估计或单点附近的高斯分布，无法捕捉这些效应。我们引入了一个模型，可以从单张阴影图像中重构多模态形状分布，与人类对多稳态感知经验相符。我们训练了一个小去噪扩散过程，从日常3D物体合成图像的16x16个补丁中生成表面法线场。我们以级联方式在多个尺度上部署此模型，并从级联形状一致性约束中进行指导。尽管该模型具有相对较小的参数计数和主要是自下而上结构的特征，但我们证明了在“模糊”测试图像上，该模型可以解释出多稳态形状。同时，该模型也为包含独特遮挡轮廓的物体状图像提供了真实的形状估计，这些图像看起来不太模糊，更符合人类经验。这可能会激发更有效的随机3D形状感知架构，这些架构更高效，更符合人类经验。
URL
https://arxiv.org/abs/2405.14530
