知识卡片

普通读书笔记卡 · 7005.k · 生成模型:DeepSeek-V4.1-Flash

训练更新风险:验证集复用与预测漂移

理解等级:中级

多数模型会在生命周期里反复更新:每几月一批数据、每周一批标注、甚至流场景每几分钟刷新。平均看每次更新应变好,个别却会变差甚至崩掉。原因之一是过拟合:每次重训要用留出验证数据检查,但若一直复用同一份验证集,反而隐式对其过拟合,故须定期刷新。另一隐患是「缺乏稳定性」:两版整体准确率相近,却可能一版擅识猫、一版擅识狗;每日重训的反欺诈模型即便 99% 精度,只要每天错的是不同 1% 用户,三个月就可能得罪全部用户。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • Basic Introduction to Models · 查看原书依据

      Each time a model is retrained or updated, we need to check for overfitting by using held-out validation data. But if we consistently reuse the same validation data, there is a risk that we may end up implicitly overfitting to this reused validation data. For this reason, it is important to refresh validation data on a regular basis and ensure we are not fooling ourselves.

    参考来源

    - 位置:《Reliable Machine Learning》第03章《Basic Introduction to Models》(源文件:extracted/Reliable Machine Learning/_epub-src/OEBPS/ch03.xhtml) - 结论依据:原文明确阐述该主题,并给出定义、实践或反例。 - 原始内容:Each time a model is retrained or updated, we need to check for overfitting by using held-out validation data. But if we consistently reuse the same validation data, there is a risk that we may end up implicitly overfitting to this reused validation data. For this reason, it is important to refresh validation data on a regular basis and ensure we are not fooling ourselves.