← 返回刚才阅读的位置

知识卡片

结构图卡 · 6750.c · 生成模型:GPT-6

翻译训练与生成使用不同目标前缀

理解等级:中级

结构图:

flowchart TD
 A[对齐源目标及匹配词表] --> B[源嵌入位置编码及padding mask]
 B --> C[源编码memory]
 D[训练真目标右移前缀] --> E[因果目标解码并交叉读取memory]
 C --> E
 E --> F[对齐后继标签并忽略PAD损失]
 G[生成从SOS起用已选前缀] --> H[同一模型逐步解码]
 C --> H
 H --> I[检查EOS或预算再决定继续]

真目标只在训练提供,生成时不能偷看尚未生成的后续词,源memory可复用。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • · 查看原书依据

      The train() function implements a training loop similar to others you have seen. Remember to put the model into train mode before the batch iteration, as shown in listing 9.17. Also, note that the last token in the target, which is the EOS token, is stripped from trg before passing it as input to the model. We want the model to predict the end of

    参考来源

    - 位置:《Natural Language Processing in Action, Second Edition》第09章《可堆叠的Transformer》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Natural Language Processing in Action, Second Edition/_epub-src/OEBPS/Text/chapter-9.html) - 结论依据:摘录中关于翻译训练与生成使用不同目标前缀的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。 - 原始内容:The train() function implements a training loop similar to others you have seen. Remember to put the model into train mode before the batch iteration, as shown in listing 9.17. Also, note that the last token in the target, which is the EOS token, is stripped from trg before passing it as input to the model. We want the model to predict the end of