← 返回刚才阅读的位置

知识卡片

普通读书笔记卡 · 7012.g · 生成模型:DeepSeek-V4.1-Flash

checkpoint 即小型自动发布与分阶段验证

理解等级:高级

多数持续 ML 设定里,模型的重大变更称为发布(launch)——架构变更、增删特征、改学习率等超参、以及其他需要在作为生产模型上线前完整重评性能的变更;基于新到数据对内部权重的小修改称为更新(update)。随模型更新会周期性写出保存当前状态的 checkpoint,推到服务,同时也是灾难恢复的重要依据。关键洞见:把新 checkpoint 推到服务其实是一次小型的自动模型发布;若每小时推四次,就接近每天上百次小型自动发布。重大发布可能出的一切错这里都可能出——模型被 bug 或用坏数据训练而损坏、模型文件被写错误/硬件 bug 甚至宇宙射线破坏、深度模型最近一步「爆炸」使参数含 NaN 或遇梯度消失而停止学习、自动化 checkpoint/推送系统自身有 bug;而若 checkpoint 与推送不自动化、靠人工,系统大概还没准备好以持续模式运行。作者澄清重点不是别更新模型(不按新数据定期更新同样会出很多问题),而是 checkpoint 推送前的验证是关键步骤:典型做法是分阶段验证——先做不影响生产的离线测试(沙箱里加载 checkpoint、对黄金集打分,第 5 章离线评估方法都适用),再把新 checkpoint 装进被仔细观察的单实例金丝雀、服务极小流量,只要监控站得住就缓慢放量直到 100%。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • Continuous ML · 查看原书依据

      One way to think about pushing a new model checkpoint out to serving is that it is actually a small, automated model launch, and if we push a new checkpoint four times an hour, then we are doing almost a hundred small, automated model launches a day. All of the things that can go wrong with a major model launch can also go wrong when we push a new checkpoint to serving.

    参考来源

    - 位置:《Reliable Machine Learning》第10章《Continuous ML》(源文件:extracted/Reliable Machine Learning/_epub-src/OEBPS/ch10.xhtml) - 结论依据:原文明确阐述该主题,并给出定义、实践或反例。 - 原始内容:One way to think about pushing a new model checkpoint out to serving is that it is actually a small, automated model launch, and if we push a new checkpoint four times an hour, then we are doing almost a hundred small, automated model launches a day. All of the things that can go wrong with a major model launch can also go wrong when we push a new checkpoint to serving.