Abstract
Artistic style transfer is concerned with the generation of imagery that combines the content of an image with the style of an artwork. In the realm of computer games, most work has focused on post-processing video frames. Some recent work has integrated style transfer into the game pipeline, but it is limited to single styles. Integrating an arbitrary style transfer method into the game pipeline is challenging due to the memory and speed requirements of games. We present PQDAST, the first solution to address this. We use a perceptual quality-guided knowledge distillation framework and train a compressed model using the FLIP evaluator, which substantially reduces both memory usage and processing time with limited impact on stylisation quality. For better preservation of depth and fine details, we utilise a synthetic dataset with depth and temporal considerations during training. The developed model is injected into the rendering pipeline to further enforce temporal stability and avoid diminishing post-process effects. Quantitative and qualitative experiments demonstrate that our approach achieves superior performance in temporal consistency, with comparable style transfer quality, to state-of-the-art image, video and in-game methods.
Abstract (translated)
艺术风格转换旨在生成结合了图像内容与艺术品风格的影像。在计算机游戏中，大多数工作集中在对视频帧进行后处理上。尽管一些近期的研究尝试将风格转移集成到游戏流水线中，但这些方法仅限于单一风格的应用。由于游戏对内存和速度的需求较高，因此将任意风格转换方法融入游戏流水线是一个挑战性的课题。我们提出了PQDAST（感知质量引导的知识蒸馏框架），这是首个解决这一问题的方案。 通过使用感知质量指导的知识蒸馏框架，并利用FLIP评估器训练一个压缩模型，我们的方法显著降低了内存消耗和处理时间，同时对风格化效果的影响有限。为了更好地保留深度信息与细节特征，在训练过程中我们采用了一个合成数据集，并考虑了深度和时间因素。开发的模型被注入到渲染流水线中，以进一步增强时间稳定性并避免减弱后期处理效果。 通过定量和定性的实验验证，我们的方法在时间一致性上表现出了优越性，同时风格转移质量与现有图像、视频及游戏内方法相当甚至更优。
URL
https://arxiv.org/abs/2502.16996
