知识卡片
数据摄取系统:把原始数据变为特征值
数据摄取系统要读取原始数据、运行特征提取代码、把结果写入特征存储。一次性、即使数据量很大的抽取可以用一次性的临时代码;但只要用户众多或反复使用,就应把它建成按需、可重复、可监控的生产流水线。其中最大的变量是用户编写的特征代码,因此平台要提供可靠环境:特征应版本化,让定义变更过渡清晰;要有测试系统校验抽取代码的基本正确性;要有 staging 环境在小样本上试跑并辅以分析工具;必要时加人工评审。若标签可由摄取时的日志关联算出,摄取系统可同时产出特征与标签并存到同一存储。
这张卡片看不懂?
这张卡片已通过人工审查,不再接受失败反馈。
已有反馈 · 0 条
还没有人提交反馈。
在整书笔记中的位置
原书出处 · 1 处
- Feature and Training Data · 查看原书依据
Data ingestion system We will have to write software that reads raw data, applies our feature extraction code to that data, and stores the resulting feature values in the feature store. In the case of one-time extraction, even for a very large amount of data, this may be a relatively ad hoc process with code designed to be run once. But in many cases, the process of extracting data is a separate production system all its own.
参考来源
- 位置:《Reliable Machine Learning》第04章《Feature and Training Data》(源文件:extracted/Reliable Machine Learning/_epub-src/OEBPS/ch04.xhtml)
- 结论依据:原文明确阐述该主题,并给出定义、实践或反例。
- 原始内容:Data ingestion system We will have to write software that reads raw data, applies our feature extraction code to that data, and stores the resulting feature values in the feature store. In the case of one-time extraction, even for a very large amount of data, this may be a relatively ad hoc process with code designed to be run once. But in many cases, the process of extracting data is a separate production system all its own.