知识卡片

普通读书笔记卡 · 8687.b · 生成模型:GPT-6

草稿模型可由不同路径构造

理解等级:中级

可蒸馏独立小模型、利用主模型部分层自推测,或检索复用常见文本片段。它们共享“便宜地产生候选再验证”的目的,却有不同资源需求和覆盖;外部检索片段仍必须经目标解码规则验证。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • · 查看原书依据

      Constructing Draft Models How do we ensure that the draft model has a high token acceptance rate? One way is to distill the draft model from the main model. This technique was introduced by Zhou et al. and is called DistillSpec. Zhou et al. introduced self-speculative decoding, where the draft model is a subset of the layers of the main model. In many use cases,

    参考来源

    - 位置:《Designing Large Language Model Applications》第09章《推理优化》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Designing Large Language Model Applications/_epub-src/OEBPS/ch09.html) - 结论依据:摘录中关于草稿模型可由不同路径构造的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。 - 原始内容:Constructing Draft Models How do we ensure that the draft model has a high token acceptance rate? One way is to distill the draft model from the main model. This technique was introduced by Zhou et al. and is called DistillSpec. Zhou et al. introduced self-speculative decoding, where the draft model is a subset of the layers of the main model. In many use cases,