知识卡片
循环并行主要沿批次而非任意时间
同一序列的后续隐藏状态依赖前一步,限制普通RNN在时间轴上的并行;不同序列和部分矩阵运算仍可批量执行。CNN不同窗口则常可并行,比较速度也须固定序列长度、架构及硬件。
这张卡片看不懂?
这张卡片已通过人工审查,不再接受失败反馈。
已有反馈 · 0 条
还没有人提交反馈。
在整书笔记中的位置
原书出处 · 1 处
- · 查看原书依据
Backpropagation for RNNs is a lot more work than for CNNs. The reason training an RNN is so computationally expensive is that it must perform the forward and backward calculations many times for each text example—once for each token in the text. And then, it has to do all that again for the next layer in the RNN. This sequence of operations is really important
参考来源
- 位置:《Natural Language Processing in Action, Second Edition》第08章《循环网络与长短期记忆》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Natural Language Processing in Action, Second Edition/_epub-src/OEBPS/Text/chapter-8.html)
- 结论依据:摘录中关于循环并行主要沿批次而非任意时间的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。
- 原始内容:Backpropagation for RNNs is a lot more work than for CNNs. The reason training an RNN is so computationally expensive is that it must perform the forward and backward calculations many times for each text example—once for each token in the text. And then, it has to do all that again for the next layer in the RNN. This sequence of operations is really important