知识卡片
MapReduce分治大数据
Map对独立数据分片并行过滤和产生键值,基础设施按键重分组,再由Reduce执行同组分析。无状态Map减少彼此通信,分片也缩小单任务失败影响。数据量小、分片大小严重不均或需多轮Reduce时,调度、洗牌和协调成本可能抵消并行收益。
结构图:
flowchart LR
A[输入分片] --> B[并行Map]
B --> C[按键洗牌分组]
C --> D[并行Reduce]
D --> E[汇总输出]
这张卡片看不懂?
这张卡片的反馈已满 10 条,暂时无法继续提交。
已有反馈 · 0 / 10 条
还没有人提交反馈。
在整书笔记中的位置
- · 第09章 性能
原书出处 · 1 处
- 性能 · 查看原书依据
The map instances are stateless and do not communicate with each other. The only communication between the map instances and the reduce instances is the data emitted from the map instances as <key, value> pairs.
参考来源
- 位置:《Software Architecture in Practice, 4th Edition》第09章《性能》(来源项目:book-sources;源文件:extracted/Software Architecture in Practice, 4th Edition/_epub-src/ch09.html)
- 结论依据:所引连续原文给出本卡机制或取舍;条件与示例结合本节已读上下文解释。
- 原始内容:The map instances are stateless and do not communicate with each other. The only communication between the map instances and the reduce instances is the data emitted from the map instances as <key, value> pairs.