← 返回刚才阅读的位置

知识卡片

普通读书笔记卡 · 8937.c · 生成模型:GPT-6

截断反传改变学习依赖

理解等级:中级

只在有限时间窗口内反向传播,可降低保存中间状态的内存与计算,但切断了窗口之前的梯度依赖。前向状态仍可能携带更早的信息,因此“有历史状态”不意味着训练能把当前损失归因到所有早期输入。需要区分状态延续和梯度截断。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • · 查看原书依据

      This approach yields exact gradients of the loss with respect to the parameters when T is equal to the total number of timesteps the RNN was run for. Otherwise, BPTT is an approximation that truncates the true computation graph. Essentially, BPTT ignores any dependencies in the computation graph greater than length T.

    参考来源

    - 位置:《Foundations of Computer Vision (Antonio Torralba, Phillip Isola etc.)》第25章《循环神经网络》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Foundations of Computer Vision (Antonio Torralba, Phillip Isola etc.)/_epub-src/OPS/xhtml/c025.xhtml) - 结论依据:摘录中关于截断反传改变学习依赖的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。 - 原始内容:This approach yields exact gradients of the loss with respect to the parameters when T is equal to the total number of timesteps the RNN was run for. Otherwise, BPTT is an approximation that truncates the true computation graph. Essentially, BPTT ignores any dependencies in the computation graph greater than length T.