知识卡片
自回归文本按条件逐项生成
解码器根据此前token预测下一个词表项,再将生成项加入上下文。一次输出是词表上的分布,而非必然一个完整单词;生成长度、终止符及采样策略会影响句子,训练时还需要正确的目标移位与因果掩码。
这张卡片看不懂?
这张卡片已通过人工审查,不再接受失败反馈。
已有反馈 · 0 条
还没有人提交反馈。
在整书笔记中的位置
- · 视觉与语言
原书出处 · 1 处
- · 查看原书依据
These steps map text to tokens, which can be input into transformers. To handle the output side, where we produce text as an output, we need to convert tokens back to text. The most common way to do this is to use an autoregressive model that predicts each next item in our vocabulary (word/character/byte-pair) given the sequence generated so far. Each prediction step can be
参考来源
- 位置:《Foundations of Computer Vision (Antonio Torralba, Phillip Isola etc.)》第51章《视觉与语言》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Foundations of Computer Vision (Antonio Torralba, Phillip Isola etc.)/_epub-src/OPS/xhtml/c051.xhtml)
- 结论依据:摘录中关于自回归文本按条件逐项生成的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。
- 原始内容:These steps map text to tokens, which can be input into transformers. To handle the output side, where we produce text as an output, we need to convert tokens back to text. The most common way to do this is to use an autoregressive model that predicts each next item in our vocabulary (word/character/byte-pair) given the sequence generated so far. Each prediction step can be