← 返回刚才阅读的位置

知识卡片

结构图卡 · 6741.a · 生成模型:GPT-6

两类门控以不同状态路径保留信息

理解等级:中级

结构图:

flowchart LR
 A[当前输入及旧h] --> B[GRU重置及更新门]
 B --> C[候选n]
 C --> D[混合候选与旧h成新h]
 A --> E[LSTM输入遗忘输出门及候选g]
 F[旧c] --> G[遗忘保留加新写入成新c]
 E --> G
 G --> H[输出门乘tanh新c成新h]

门值为连续控制量,状态路径有助长依赖训练,但没有无限无损记忆保证。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • · 查看原书依据

      8.4.2 Gated recurrent units For short text, ordinary RNNs with a single activation function for each neuron work well. All your neurons need to do is recycle and reuse the hidden vector representation of what they have read so far in the text. But ordinary RNNs have a short attention span that limits their ability to understand longer texts. The influence of the first token

    参考来源

    - 位置:《Natural Language Processing in Action, Second Edition》第08章《循环网络与长短期记忆》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Natural Language Processing in Action, Second Edition/_epub-src/OEBPS/Text/chapter-8.html) - 结论依据:摘录中关于两类门控以不同状态路径保留信息的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。 - 原始内容:8.4.2 Gated recurrent units For short text, ordinary RNNs with a single activation function for each neuron work well. All your neurons need to do is recycle and reuse the hidden vector representation of what they have read so far in the text. But ordinary RNNs have a short attention span that limits their ability to understand longer texts. The influence of the first token