知识卡片

普通读书笔记卡 · 7009.o · 生成模型:DeepSeek-V4.1-Flash

好模型会变坏与回退计划

理解等级:高级

应当假设产出的模型难以复现、且未来会出现或大或小的可靠性问题:即便上线时表现良好,模型或世界也可能以难以预测的方式变化,带来日后的大麻烦,因此要预备备份计划。第一套是做非 ML(或至少更简单 ML)的回退/失效安全实现,用启发式、算法或默认值保证模型无法给出精细预测时应用仍有基本功能——例如 yarnit.ai 的推荐模型不可用时改为展示热门商品。代价是它限制了模型能好到什么程度:模型一旦远超启发式,依赖过深就没有任何备份能顶上;对早期采用 ML 的组织这完全合适,但要真正吃到 ML 红利就得逐步摆脱。第二套是保留多版本以便回滚,只在新版本明显变差时有用;世界整体变化使所有版本都失效时,出路是多版本并存加快速开发新变体。作者也提醒:模型开发与生产之间并不存在固定而可辩护的屏障,模型质量既是生产工程问题也是建模问题。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • Training Systems · 查看原书依据

      Good Models Will Become Bad We should assume that the models we produce will be hard to reproduce and will have subtle and large reliability problems in the future. Even when a model works well when we launch it, we have to assume that either the model or the world might change in some difficult-to-predict way that causes us enormous trouble in future years.

    参考来源

    - 位置:《Reliable Machine Learning》第07章《Training Systems》(源文件:extracted/Reliable Machine Learning/_epub-src/OEBPS/ch07.xhtml) - 结论依据:原文明确阐述该主题,并给出定义、实践或反例。 - 原始内容:Good Models Will Become Bad We should assume that the models we produce will be hard to reproduce and will have subtle and large reliability problems in the future. Even when a model works well when we launch it, we have to assume that either the model or the world might change in some difficult-to-predict way that causes us enormous trouble in future years.