← 返回刚才阅读的位置

知识卡片

普通读书笔记卡 · 9220.d · 生成模型:GPT-6

前向缓存供反向求导复用

理解等级:中级

前向记录必要的激活或输出,反向按逆依赖顺序计算梯度,避免重复计算相同子表达式。缓存与参数须对应同一次前向状态,不能在逐层求完之前就随意更新权重,导致后续梯度使用新旧混合值。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • · 查看原书依据

      This computation can be executed layer by layer, starting from the output layer and working its way backward. This phase is computationally very similar to the forward phase and allows the reuse of all the activations and outputs that have been previously computed. As an extra goody, the derivative of the sigmoid function σ′(x) = σ(x)(1−σ(x)), resulting in

    参考来源

    - 位置:《Introduction to Autonomous Robots Mechanisms, Sensors, Actuators, and Algorithms (Nikolaus Correll, Bradley Hayes etc.)》附录D《附录D:反向传播》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Introduction to Autonomous Robots Mechanisms, Sensors, Actuators, and Algorithms (Nikolaus Correll, Bradley Hayes etc.)/_epub-src/OEBPS/xhtml/appendixD.xhtml) - 结论依据:摘录中关于前向缓存供反向求导复用的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。 - 原始内容:This computation can be executed layer by layer, starting from the output layer and working its way backward. This phase is computationally very similar to the forward phase and allows the reuse of all the activations and outputs that have been previously computed. As an extra goody, the derivative of the sigmoid function σ′(x) = σ(x)(1−σ(x)), resulting in