← 返回刚才阅读的位置

知识卡片

普通读书笔记卡 · 7004.i · 生成模型:DeepSeek-V4.1-Flash

清洗与归一化

理解等级:中级

清洗与一致性最好集中到单一可审查、可改进的地方,而非散落训练管线各处——否则假设难以保证、改进难以同步,甚至对同一数据多次“纠正”抹掉原信息或产生竞态。归一化把数据变换到相近尺度以利梯度下降等优化,常用:缩放到范围、截断(clipping)、对数缩放、Z-score。危险在于:若范围/分布/均值在训练集与测试集属性不同,这些变换会出错;分桶(bucketing)不同边界策略更是难检错误的来源,须保留原字段并写出新字段以便将来切换。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • Data Management Principles · 查看原书依据

      It is important to note that each of these techniques can be dangerous if any of the range or distribution or means are calculated on a set of test data with different properties than the dataset where it is later applied. A final related and common technique is that of putting the data into buckets: we map a range of data into a much smaller set of groups that represent the same range.

    参考来源

    - 位置:《Reliable Machine Learning》第02章《Data Management Principles》(源文件:extracted/Reliable Machine Learning/_epub-src/OEBPS/ch02.xhtml) - 结论依据:原文主张集中清洗,并列出归一化技术及其在跨分布计算时的风险。 - 原始内容:It is important to note that each of these techniques can be dangerous if any of the range or distribution or means are calculated on a set of test data with different properties than the dataset where it is later applied. A final related and common technique is that of putting the data into buckets: we map a range of data into a much smaller set of groups that represent the same range.