知识卡片
抽样演示不等于多词生成
从同一个下一词分布有放回抽取一千次,是观察该位置频率的实验。真正连续生成每次只选一个 token,再更新上下文得到新的分布;不能把同一分布的独立抽样拼接成具有相同条件依赖的回答。
这张卡片看不懂?
这张卡片已通过人工审查,不再接受失败反馈。
已有反馈 · 0 条
还没有人提交反馈。
在整书笔记中的位置
- · 推理时计算扩展
原书出处 · 1 处
- · 查看原书依据
meant for illustration only. It draws many samples from the distribution so you can see how often each token is selected. In real text generation we only draw one token at a time, but taking a large number of samples here makes the underlying probabilities easier to visualize and understand.
参考来源
- 位置:《Build a Reasoning Model (From Scratch)》第04章《推理时计算扩展》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Build a Reasoning Model (From Scratch)/_epub-src/OEBPS/Text/chapter-4.html)
- 结论依据:摘录中关于抽样演示不等于多词生成的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。
- 原始内容:meant for illustration only. It draws many samples from the distribution so you can see how often each token is selected. In real text generation we only draw one token at a time, but taking a large number of samples here makes the underlying probabilities easier to visualize and understand.