知识卡片
流式输出仍需统一解码
用 yield 暴露每一步 token 可降低等待整段完成的体验延迟,却不会让自回归依赖自动并行。对字节级子词直接逐 token 解码可能遇到跨 token 字符边界,应使用能保持解码状态的方式或检查累积解码前缀;同时保留 EOS 和预算逻辑。
这张卡片看不懂?
这张卡片已通过人工审查,不再接受失败反馈。
已有反馈 · 0 条
还没有人提交反馈。
在整书笔记中的位置
原书出处 · 1 处
- · 查看原书依据
Exercise 2.2: Streaming token generation Write a modified version of the generate_text_basic function that returns each token as it is generated and prints it, which is also known as streaming token generation. The goal of this exercise is to understand how to implement token-by-token text generation, a technique often used in real-time applications like chatbots and interactive assistants. Tip 1: Use yield instead of return
参考来源
- 位置:《Build a Reasoning Model (From Scratch)》第02章《使用预训练模型生成文本》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Build a Reasoning Model (From Scratch)/_epub-src/OEBPS/Text/chapter-2.html)
- 结论依据:摘录中关于流式输出仍需统一解码的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。
- 原始内容:Exercise 2.2: Streaming token generation Write a modified version of the generate_text_basic function that returns each token as it is generated and prints it, which is also known as streaming token generation. The goal of this exercise is to understand how to implement token-by-token text generation, a technique often used in real-time applications like chatbots and interactive assistants. Tip 1: Use yield instead of return