知识卡片

普通读书笔记卡 · 6733.b · 生成模型:GPT-6

时间反传按共享使用位置累计梯度

理解等级:中级

展开计算图后,链式法则把损失对每个时间步状态的影响传回,并累加到共享参数。通常先求完所需梯度再由优化器更新一次;不是先逐词修改权重,再用已改权重继续同一次反传。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • · 查看原书依据

      8.3 Backpropagation through time Backpropagation for RNNs is a lot more work than for CNNs. The reason training an RNN is so computationally expensive is that it must perform the forward and backward calculations many times for each text example—once for each token in the text. And then, it has to do all that again for the next layer in the RNN. This sequence of

    参考来源

    - 位置:《Natural Language Processing in Action, Second Edition》第08章《循环网络与长短期记忆》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Natural Language Processing in Action, Second Edition/_epub-src/OEBPS/Text/chapter-8.html) - 结论依据:摘录中关于时间反传按共享使用位置累计梯度的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。 - 原始内容:8.3 Backpropagation through time Backpropagation for RNNs is a lot more work than for CNNs. The reason training an RNN is so computationally expensive is that it must perform the forward and backward calculations many times for each text example—once for each token in the text. And then, it has to do all that again for the next layer in the RNN. This sequence of