知识卡片

普通读书笔记卡 · 7007.h · 生成模型:DeepSeek-V4.1-Flash

黄金数据集

理解等级:高级

持续重训并用渐进式验证的系统里,很难分清是模型变差了,还是数据本身变得更容易或更难预测。黄金数据集是解法:取某个固定时间点(如去年 10 月 1 日)的一批数据,明令永不用于训练,并连同当时的模型输出一起封存,甚至把这些结果视为“正确”。由于新训练数据只会让模型在黄金集上表现缓慢变化,一旦指标突变就极有利于定位 bug。局限也明确:它不适合判断绝对质量,相关性随时间衰减;数据隐私法规或删除、过期要求也可能让它无法长期保留。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • Evaluating Model Validity and Quality · 查看原书依据

      One way to control this is to create a golden set of data that models are not ever allowed to train on, but that is from a specific point in time. For example, we might decide that the data from October 1 of last year might be set aside as golden set data, never to be used for training under any circumstance, but held aside.

    参考来源

    - 位置:《Reliable Machine Learning》第05章《Evaluating Model Validity and Quality》(源文件:extracted/Reliable Machine Learning/_epub-src/OEBPS/ch05.xhtml) - 结论依据:原文明确阐述该主题,并给出定义、实践或反例。 - 原始内容:One way to control this is to create a golden set of data that models are not ever allowed to train on, but that is from a specific point in time. For example, we might decide that the data from October 1 of last year might be set aside as golden set data, never to be used for training under any circumstance, but held aside.