知识卡片

普通读书笔记卡 · 2280.q.15

数据湖延后转换仍有耦合

理解等级:高级

数据湖保留集中存储与数据管道,但先加载近原始数据,消费者按实际用途再转换,避免仓库提前做大量无用建模,也更适合部分机器学习场景。它没有自动保留领域语义:发现关联、质量校验与敏感信息识别仍需专家,转换工作只是下移。上游变化、批量陈旧和管道脆弱性仍存在,原始数据的组合还可能暴露隐私。

这张卡片看不懂?

已有反馈 · 0 / 10 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • 管理分析数据 · 查看原书依据

      As in many reactionary responses to the complexity, expense, and failures of the data warehouse, the design pendulum swung to the opposite pole, exemplified by the Data Lake pattern, intentionally the inverse of the Data Warehouse pattern. While it keeps the centralized model and pipelines, it inverts the “transform and load” model of the data warehouse to a “load and transform” one.

    参考来源

    - 位置:《Software Architecture The Hard Parts Modern Trade-Off Analyses for Distributed Architectures (Neal Ford, Mark Richards, Pramod Sadalage etc.)》第14章《Chapter 14. Managing Analytical Data》(来源项目:book-sources;源文件:extracted/Software Architecture The Hard Parts Modern Trade-Off Analyses for Distributed Architectures (Neal Ford, Mark Richards, Pramod Sadalage etc.)/_epub-src/OEBPS/ch14.xhtml) - 结论依据:原文的定义、示例或失败路径支持以下归纳:数据湖保留集中存储与数据管道,但先加载近原始数据,消费者按实际用途再转换,避免仓库提前做大量无用建模,也更适合部分机器学习场景。 其他条件结合本章同小节说明;不将模式目标当成已有运行保证。 - 原始内容:As in many reactionary responses to the complexity, expense, and failures of the data warehouse, the design pendulum swung to the opposite pole, exemplified by the Data Lake pattern, intentionally the inverse of the Data Warehouse pattern. While it keeps the centralized model and pipelines, it inverts the “transform and load” model of the data warehouse to a “load and transform” one.