知识卡片
加性细胞通路缓解消失
若遗忘门接近1、输入门抑制新写入,细胞状态可沿近似恒等的加性通路传播,减弱反复非线性变换造成的梯度衰减。直接细胞路径的偏导含f,但完整梯度还包含门与隐藏态的依赖,因此LSTM不是对梯度消失或爆炸的无条件免疫。
这张卡片看不懂?
这张卡片已通过人工审查,不再接受失败反馈。
已有反馈 · 0 条
还没有人提交反馈。
在整书笔记中的位置
- · 循环神经网络
原书出处 · 1 处
- · 查看原书依据
The basic idea is it should be easy, by default, not to forget. The cell state controls what is remembered. It gets modulated, but ft can learn to be all ones so that information propagates forward through an identity connection from the previous cell state to the subsequent cell state. This idea is similar to the idea of skip connections and residual connections in U-Nets
参考来源
- 位置:《Foundations of Computer Vision (Antonio Torralba, Phillip Isola etc.)》第25章《循环神经网络》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Foundations of Computer Vision (Antonio Torralba, Phillip Isola etc.)/_epub-src/OPS/xhtml/c025.xhtml)
- 结论依据:摘录中关于加性细胞通路缓解消失的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。
- 原始内容:The basic idea is it should be easy, by default, not to forget. The cell state controls what is remembered. It gets modulated, but ft can learn to be all ones so that information propagates forward through an identity connection from the previous cell state to the subsequent cell state. This idea is similar to the idea of skip connections and residual connections in U-Nets