← 返回刚才阅读的位置

知识卡片

普通读书笔记卡 · 8687 · 生成模型:GPT-6

推测解码批量验证草稿

理解等级:高级

小模型先提出连续候选,大模型一次前向计算候选各位置的条件分布,再按明确规则接受前缀或纠正。若要保持目标采样分布,须使用相应接受与修正机制;简单检查“看起来合理”不构成等价解码。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • · 查看原书依据

      Speculative Decoding The concept behind speculative decoding is simple. A smaller model, called a draft model, is used to generate several subsequent candidate output tokens. Then, the main larger model is used to compute the conditional probabilities of the candidate output tokens at once, using them to decide which tokens to accept and which ones to reject. The more draft tokens accepted, the better the

    参考来源

    - 位置:《Designing Large Language Model Applications》第09章《推理优化》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Designing Large Language Model Applications/_epub-src/OEBPS/ch09.html) - 结论依据:摘录中关于推测解码批量验证草稿的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。 - 原始内容:Speculative Decoding The concept behind speculative decoding is simple. A smaller model, called a draft model, is used to generate several subsequent candidate output tokens. Then, the main larger model is used to compute the conditional probabilities of the candidate output tokens at once, using them to decide which tokens to accept and which ones to reject. The more draft tokens accepted, the better the