知识卡片
管道分离输入转换输出
把管道组织为可复用、版本化的转换步骤,明确输入、逻辑和输出,有助于复现与并行处理。消费方仍需检验符合本用例的质量、安全与完整性,不能因为上游产品通过验证就取消所有本地检查。
这张卡片看不懂?
这张卡片已通过人工审查,不再接受失败反馈。
已有反馈 · 0 条
还没有人提交反馈。
在整书笔记中的位置
原书出处 · 1 处
- Turning Data into Value · 查看原书依据
I recommend assembling your data pipelines as a series of isolated, immutable transformations that can be reused and combined easily. Input, transformation logic, and output in this case must be clearly separated from each other. For reproducible output data, I recommend versioning all pipelines. For better performance that takes advantage of the elasticity of modern infrastructure, you might want to engineer your pipelines to run
参考来源
- 位置:《Data Management at Scale》第11章《Turning Data into Value》(来源项目 a19c7933-4eda-4176-aebd-30fe7e1ad292,源文件:extracted/Data Management at Scale/_epub-src/OEBPS/ch11.html)
- 结论依据:原文对isolated, immutable transformations的解释、约束及上下文支持本卡;内容为中文归纳,不把架构建议当作所有组织的必然规律。
- 原始内容:I recommend assembling your data pipelines as a series of isolated, immutable transformations that can be reused and combined easily. Input, transformation logic, and output in this case must be clearly separated from each other. For reproducible output data, I recommend versioning all pipelines. For better performance that takes advantage of the elasticity of modern infrastructure, you might want to engineer your pipelines to run