← 返回刚才阅读的位置

知识卡片

结构图卡 · 6741 · 生成模型:GPT-6

循环计算与训练更新分属两种时间

理解等级:中级

结构图:

flowchart TD
 A[固定参数及样本初始状态] --> B[当前输入x_t与旧状态]
 B --> C[同一转移函数计算新状态]
 C --> D[按任务输出当前或最终预测]
 C --> E{样本尚有输入}
 E -->|是| B
 D --> F[汇总所需损失]
 F --> G[沿时间与层反向累计梯度]
 G --> H[优化器更新共享参数]

状态逐词改变,权重在优化步骤改变;不能把隐藏激活变化叫每个词重新训练参数。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • · 查看原书依据

      8.3 Backpropagation through time Backpropagation for RNNs is a lot more work than for CNNs. The reason training an RNN is so computationally expensive is that it must perform the forward and backward calculations many times for each text example—once for each token in the text. And then, it has to do all that again for the next layer in the RNN. This sequence of

    参考来源

    - 位置:《Natural Language Processing in Action, Second Edition》第08章《循环网络与长短期记忆》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Natural Language Processing in Action, Second Edition/_epub-src/OEBPS/Text/chapter-8.html) - 结论依据:摘录中关于循环计算与训练更新分属两种时间的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。 - 原始内容:8.3 Backpropagation through time Backpropagation for RNNs is a lot more work than for CNNs. The reason training an RNN is so computationally expensive is that it must perform the forward and backward calculations many times for each text example—once for each token in the text. And then, it has to do all that again for the next layer in the RNN. This sequence of