Abstract
Prompt-driven image analysis converts a single natural-language instruction into multiple steps: locate, segment, edit, and describe. We present a practical case study of a unified pipeline that combines open-vocabulary detection, promptable segmentation, text-conditioned inpainting, and vision-language description into a single workflow. The system works end to end from a single prompt, retains intermediate artifacts for transparent debugging (such as detections, masks, overlays, edited images, and before and after composites), and provides the same functionality through an interactive UI and a scriptable CLI for consistent, repeatable runs. We highlight integration choices that reduce brittleness, including threshold adjustments, mask inspection with light morphology, and resource-aware defaults. In a small, single-word prompt segment, detection and segmentation produced usable masks in over 90% of cases with an accuracy above 85% based on our criteria. On a high-end GPU, inpainting makes up 60 to 75% of total runtime under typical guidance and sampling settings, which highlights the need for careful tuning. The study offers implementation-guided advice on thresholds, mask tightness, and diffusion parameters, and details version pinning, artifact logging, and seed control to support replay. Our contribution is a transparent, reliable pattern for assembling modern vision and multimodal models behind a single prompt, with clear guardrails and operational practices that improve reliability in object replacement, scene augmentation, and removal.
Abstract (translated)
提示驱动的图像分析将自然语言指令转换为多个步骤：定位、分割、编辑和描述。我们提出了一种实用案例研究，展示了一个统一的工作流程管道，该管道结合了开放词汇检测、可提示分割、基于文本条件的修复（即在已有内容基础上生成缺失或损坏部分的新内容）、以及视觉-语言描述等功能。系统从单个提示开始，并且在整个过程中保留中间产物以实现透明调试（例如：检测结果、掩码、叠加层、编辑后的图像和前后对比合成）。该系统还通过交互式UI和脚本化CLI提供了相同的功能，确保一致性和可重复性运行。 我们强调了减少脆弱性的集成选择，如调整阈值、使用轻量级形态学进行掩模检查以及资源感知的默认设置。在小规模的一词提示分割实验中，检测与分割产生的可用掩码超过90%的情况达到了85%以上的准确率。在高端GPU上，在典型的指导和采样设定下，修复操作占据了总运行时间的60至75%，这强调了仔细调优的重要性。 这项研究提供了关于阈值、掩模紧致度以及扩散参数等方面的实施建议，并详细说明了版本固定、日志记录和种子控制以支持回放。我们的贡献在于提供了一种透明可靠的模式，用于在单个提示背后组装现代视觉及多模态模型，包括明确的保护措施和操作实践，这些有助于提高对象替换、场景增强以及移除等任务的可靠性。
URL
https://arxiv.org/abs/2509.08489
