知识卡片

普通读书笔记卡 · 6733.c · 生成模型:GPT-6

长时间链会消失或放大梯度

理解等级:中级

反向沿时间反复乘导数与状态变换,可能使远处贡献很小或数值爆炸。门控与结构设计可改善传播,训练仍需监控,必要时采用相应截断或梯度控制;能够画出任意长链不代表可稳定学会无限长依赖。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • · 查看原书依据

      Because RNNs are particularly deep neural nets, RNN gradients are particularly temperamental, and they may disappear or explode. Efficiently modeling natural language character sequences wasn’t possible until recurrent neural nets were applied to the task. Weights in an RNN are adjusted in aggregate across time for a given sample. You can use different methods to examine the output of recurrent neural nets. You can model

    参考来源

    - 位置:《Natural Language Processing in Action, Second Edition》第08章《循环网络与长短期记忆》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Natural Language Processing in Action, Second Edition/_epub-src/OEBPS/Text/chapter-8.html) - 结论依据:摘录中关于长时间链会消失或放大梯度的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。 - 原始内容:Because RNNs are particularly deep neural nets, RNN gradients are particularly temperamental, and they may disappear or explode. Efficiently modeling natural language character sequences wasn’t possible until recurrent neural nets were applied to the task. Weights in an RNN are adjusted in aggregate across time for a given sample. You can use different methods to examine the output of recurrent neural nets. You can model