知识卡片

普通读书笔记卡 · 6750 · 生成模型:GPT-6

教师强制输入与标签相差一位

理解等级:中级

训练将目标序列去掉最后一项作输入,去掉第一项作预测标签,配合因果mask学习下一个token。SOS只作起点,EOS仍应在标签中受监督;若真实序列未加这些项,切片不会自动创造正确边界。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • · 查看原书依据

      The train() function implements a training loop similar to others you have seen. Remember to put the model into train mode before the batch iteration, as shown in listing 9.17. Also, note that the last token in the target, which is the EOS token, is stripped from trg before passing it as input to the model. We want the model to predict the end of

    参考来源

    - 位置:《Natural Language Processing in Action, Second Edition》第09章《可堆叠的Transformer》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Natural Language Processing in Action, Second Edition/_epub-src/OEBPS/Text/chapter-9.html) - 结论依据:摘录中关于教师强制输入与标签相差一位的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。 - 原始内容:The train() function implements a training loop similar to others you have seen. Remember to put the model into train mode before the batch iteration, as shown in listing 9.17. Also, note that the last token in the target, which is the EOS token, is stripped from trg before passing it as input to the model. We want the model to predict the end of