Abstract
Image compression emerges as a pivotal tool in the efficient handling and transmission of digital images. Its ability to substantially reduce file size not only facilitates enhanced data storage capacity but also potentially brings advantages to the development of continual machine learning (ML) systems, which learn new knowledge incrementally from sequential data. Continual ML systems often rely on storing representative samples, also known as exemplars, within a limited memory constraint to maintain the performance on previously learned data. These methods are known as memory replay-based algorithms and have proven effective at mitigating the detrimental effects of catastrophic forgetting. Nonetheless, the limited memory buffer size often falls short of adequately representing the entire data distribution. In this paper, we explore the use of image compression as a strategy to enhance the buffer's capacity, thereby increasing exemplar diversity. However, directly using compressed exemplars introduces domain shift during continual ML, marked by a discrepancy between compressed training data and uncompressed testing data. Additionally, it is essential to determine the appropriate compression algorithm and select the most effective rate for continual ML systems to balance the trade-off between exemplar quality and quantity. To this end, we introduce a new framework to incorporate image compression for continual ML including a pre-processing data compression step and an efficient compression rate/algorithm selection method. We conduct extensive experiments on CIFAR-100 and ImageNet datasets and show that our method significantly improves image classification accuracy in continual ML settings.
Abstract (translated)
图像压缩成为处理和传输数字图像的高效手段。其大幅减小文件大小不仅提高了数据存储容量，还有助于连续机器学习（ML）系统的开发，这些系统从序列数据中逐步学习新知识。连续ML系统通常需要在有限内存约束下存储代表性样本，也就是实例，以保持对之前学习数据的性能。这些方法称为基于回放的算法，已经在减轻灾难性遗忘的有害影响方面取得了有效成果。然而，有限的内存缓冲区往往无法充分表示整个数据分布。在本文中，我们探讨了将图像压缩作为一种策略来提高缓冲器容量，从而增加实例多样性。然而，直接使用压缩实例在连续ML过程中会导致领域漂移，表现为压缩训练数据和未压缩测试数据之间的差异。此外，确定适当的压缩算法以及为连续ML系统选择最有效的压缩率至关重要。为此，我们引入了一个新的框架，包括预处理数据压缩步骤和高效的压缩率/算法选择方法，用于将图像压缩应用于连续ML。我们在CIFAR-100和ImageNet数据集上进行了广泛的实验，结果表明，我们的方法在连续ML环境中显著提高了图像分类准确性。
URL
https://arxiv.org/abs/2403.06288
