← 返回刚才阅读的位置

知识卡片

普通读书笔记卡 · 7009.p · 生成模型:DeepSeek-V4.1-Flash

数据不可用的有偏性

理解等级:高级

训练用的部分数据 attempted 读取时必然不可用:分布式存储系统里真正的数据丢失很少,但数据不可用的量要高得多,所以必须提前想清楚。而训练数据集本就已是更大总体的抽样(起始采集日期之前没有数据、只含本站购买行为、高量日志还会为省钱自动子采样),因此丢数据时该问的不是丢了多少,而是「丢失是否有偏」:完全随机地每 1000 条丢 1 条,几乎肯定可忽略;丢掉整个西班牙用户、早晨购物者、或大型针织会议前一天的数据,则会在训练数据与被排除数据之间制造新偏差。想为全系统预先统一解决缺失,只在所有模型约束与目标相似时才成立。反欺诈系统还有对抗含义:攻击者会用短时海量重复交易压垮处理、试图命中高丢弃率启发式,从而系统性左右哪些时段被跳过;可靠性团队必须确保外部无法做到这一点。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • Training Systems · 查看原书依据

      Given that our training data is already subsampled, probably in several ways, when we lose data, we should answer this question: is the loss of data biased in some way? If we were to drop one out of every 1,000 training records in a completely random way, this is almost certainly safe to ignore for the model. On the other hand, losing all of the data from people in Spain, or from people who shop in the mornings, or from the da

    参考来源

    - 位置:《Reliable Machine Learning》第07章《Training Systems》(源文件:extracted/Reliable Machine Learning/_epub-src/OEBPS/ch07.xhtml) - 结论依据:原文明确阐述该主题,并给出定义、实践或反例。 - 原始内容:Given that our training data is already subsampled, probably in several ways, when we lose data, we should answer this question: is the loss of data biased in some way? If we were to drop one out of every 1,000 training records in a completely random way, this is almost certainly safe to ignore for the model. On the other hand, losing all of the data from people in Spain, or from people who shop in the mornings, or from the da