这其实更像是预训练终结的一个副作用。 随着 transformer 逐渐饱和，大多数预训练成果都只是在微调数据配方，而不是性能上大幅度的阶跃式提升。 这就把焦点转向了后训练和测试时计算上的创新。这对竞争是好事，因为预训练把资源集中在了少数几个实验室手里。
机器翻译
SemiAnalysis@semianalysis_
GLM 5.3’s biggest leap came from post-training alone. “GLM 5.3 was built entirely by post-training the previous model. For me, that was a wake-up call.” “Post-training is far more GPU-efficient than pre-training from scratch. Plenty of companies can’t afford to train a model from scratch, but they have proprietary data. They can post-train themselves and beat some of the closed models.” “There’s still so much juice left to squeeze from post-training. We compared GLM 5.3’s cyber capabilities with 5.2, and it’s a night and day difference. That’s all just post-training.”
查看被引用的帖子