Abstract
We present a design called \emph{Proof of Gradient Optimization} (PoGO) for blockchain consensus, where miners produce verifiable evidence of training large-scale machine-learning models. Building on previous work, we incorporate \emph{quantized gradients} (4-bit precision) to reduce storage and computation requirements, while still preserving the ability of verifiers to check that real progress has been made on lowering the model's loss. Additionally, we employ Merkle proofs over the full 32-bit model to handle large parameter sets and to enable random leaf checks with minimal on-chain data. We illustrate these ideas using GPT-3 (175B parameters) as a reference example and also refer to smaller but high-performance models (e.g., \emph{Gemma~3} with 27B parameters). We provide an empirical cost analysis showing that verification is significantly cheaper than training, thanks in part to quantization and sampling. We also discuss the necessity of longer block times (potentially hours) when incorporating meaningful training steps, the trade-offs when using specialized GPU hardware, and how binary diffs may incrementally optimize updates. Finally, we note that fine-tuning can be handled in a similar manner, merely changing the dataset and the manner of sampling but preserving the overall verification flow. Our protocol allows verifiers to issue either \emph{positive} or \emph{negative} attestations; these are aggregated at finalization to either confirm the update or slash the miner.
Abstract (translated)
我们提出了一种名为“梯度优化证明”(PoGO)的区块链共识设计，其中矿工生成训练大规模机器学习模型的有效证据。在此前工作的基础上，我们采用4位精度的量化梯度来减少存储和计算需求，同时仍然保持验证者检查实际损失降低进展的能力。此外，我们在整个32位模型上使用Merkle证明处理大量参数，并允许进行随机叶节点检查，仅需极少量的链上数据。我们以GPT-3（1750亿参数）作为参考示例来阐述这些想法，并提及一些较小但高性能的模型（如具有270亿参数的Gemma 3）。我们提供了一个经验成本分析，表明由于量化和采样的原因，验证比训练要便宜得多。我们还讨论了在包含有意义的训练步骤时需要更长的区块时间（可能长达数小时），使用专用GPU硬件时的成本与收益权衡，以及二进制差分如何逐步优化更新的问题。最后，我们注意到微调可以以类似的方式处理，只需更改数据集和采样的方式即可保持整体验证流程不变。我们的协议允许验证者发布“积极”或“消极”的认证；这些在最终确定时进行汇总，确认更新或削减矿工的奖励。
URL
https://arxiv.org/abs/2504.07540
