← 返回刚才阅读的位置

知识卡片

普通读书笔记卡 · 7012.e · 生成模型:DeepSeek-V4.1-Flash

特征商店、非均匀采样与原始日志

理解等级:高级

典型生产 ML 系统把原始数据转换成特征,既便于学习、存储也更紧凑;许多系统用特征商店存放——它本质是一个增强型数据库:管理输入流、知道如何把原始数据转成特征、高效存储、允许跨项目共享、并同时支撑模型训练与服务。高容量应用里常需从整体数据流做一定采样以降低存储与处理成本,而采样往往不均匀:例如我们可能希望保留全部(稀有的)正例,只留一小部分(极常见的)负例,这就必须跟踪这些采样偏置,并在训练中以适当权重把它们纳入。此外,即便特征化数据更紧凑、通常也更有用,几乎总还是必须保留一定量记录下来的原始数据:既用于开发新特征,也用于测试与验证特征提取和变换代码路径的正确性。这条「留原始日志」的要求与第 7 章特征存储的可追溯性、第 9 章特征漂移排查相互支撑——没有原始日志,就无法在事后重算或证伪一条特征变换。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • Continuous ML · 查看原书依据

      For example, we might wish to keep all of the (rare) positives but only a fraction of the (very common) negatives, which means we will need to track these sampling biases and incorporate them with appropriate weighting into training. Even though the featurized data is more compact and generally more useful, it will almost always be necessary to also keep some amount of logged raw data as well.

    参考来源

    - 位置:《Reliable Machine Learning》第10章《Continuous ML》(源文件:extracted/Reliable Machine Learning/_epub-src/OEBPS/ch10.xhtml) - 结论依据:原文明确阐述该主题,并给出定义、实践或反例。 - 原始内容:For example, we might wish to keep all of the (rare) positives but only a fraction of the (very common) negatives, which means we will need to track these sampling biases and incorporate them with appropriate weighting into training. Even though the featurized data is more compact and generally more useful, it will almost always be necessary to also keep some amount of logged raw data as well.