知识卡片

普通读书笔记卡 · 7007.f · 生成模型:DeepSeek-V4.1-Flash

IID 留出测试集

理解等级:中级

最常见的评估分布是留出测试集:随机留出一部分数据只用于评估,当每个样本独立且等概率被留出时称为 IID(独立同分布)测试集。它流行并非因为信息量最大,而是契合监督学习理论假设。但纯 IID 可能过于乐观:股价预测里 12:01 与 12:03 落在训练集、12:02 落在测试集,模型等于偷看了“未来”;真实场景下预测 12:02 拿不到 12:03 的信息。结论不是 IID 一定坏,而是构造评估数据要靠推理与常识,别套固定配方。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • Evaluating Model Validity and Quality · 查看原书依据

      If we create a purely IID test set, we might have training data from 12:01 and 12:03 from a given day in the training data, while data from 12:02 ends up in the test data. This would create a situation in which the model can make a better guess about 12:02 because it has seen the “future” of what 12:03 looks like.

    参考来源

    - 位置:《Reliable Machine Learning》第05章《Evaluating Model Validity and Quality》(源文件:extracted/Reliable Machine Learning/_epub-src/OEBPS/ch05.xhtml) - 结论依据:原文明确阐述该主题,并给出定义、实践或反例。 - 原始内容:If we create a purely IID test set, we might have training data from 12:01 and 12:03 from a given day in the training data, while data from 12:02 ends up in the test data. This would create a situation in which the model can make a better guess about 12:02 because it has seen the “future” of what 12:03 looks like.