Abstract
State-of-the-art (SOTA) generative models have demonstrated impressive capabilities in image synthesis or text generation, often with a large capacity model. However, these large models cannot be deployed on smartphones due to the limited availability of on-board memory and computations. Quantization methods lower the precision of the model parameters, allowing for efficient computations, \eg, in \INT{8}. Although aggressive quantization addresses efficiency and memory constraints, preserving the quality of the model remains a challenge. To retain quality in previous aggressive quantization, we propose a new fractional bits quantization (\short) approach. The novelty is a simple yet effective idea: we progressively reduce the model's precision from 32 to 4 bits per parameter, and exploit the fractional bits during optimization to maintain high generation quality. We show that the \short{} yields improved quality on a variety of diffusion models, including SD3.5-Medium, Sana, \pixart, and FLUX.1-schnell, while achieving $4-7\%$ lower FiD than standard QAT. Finally, we deploy and run Sana on a Samsung S25U, which runs on the Qualcomm SM8750-AB Snapdragon 8 Elite Hexagon Tensor Processor (HTP).
Abstract (translated)
最先进的（SOTA）生成模型在图像合成或文本生成方面表现出令人印象深刻的能力，通常使用的是大规模模型。然而，由于手机上的内存和计算资源有限，这些大型模型无法部署到智能手机上。量化方法通过降低模型参数的精度来实现高效的计算，例如采用INT8格式。尽管激进的量化方法可以解决效率和内存限制问题，但保持模型质量仍是一个挑战。为了在之前的激进量化中保留质量，我们提出了一种新的分数位量化（简称FracBits）方法。其新颖之处在于一个简单而有效的想法：我们逐步将模型精度从32位减少到每参数4位，并利用分数位进行优化以保持高质量的生成效果。我们展示了FracBits在多种扩散模型上的改进，包括SD3.5-Medium、Sana、PixArt和FLUX.1-schnell，在FiD指标上比标准QAT低4-7%。最后，我们在三星S25U设备（搭载高通SM8750-AB骁龙8 Elite Hexagon Tensor Processor (HTP)）上部署并运行了Sana模型。 这一段翻译概述了一种新的分数位量化方法，旨在解决大型生成模型在手机等资源受限的设备上的部署问题。该方法通过逐步减少模型参数的精度，并利用分数位进行优化，在保持高质量的同时实现了更高效的计算。
URL
https://arxiv.org/abs/2510.14823
