← 返回刚才阅读的位置

知识卡片

普通读书笔记卡 · 8686.b · 生成模型:GPT-6

蒸馏扩增要检查输入多样性

理解等级:中级

让教师参照少量种子同时生成输入与输出,可扩充任务数据。输入也由教师产生,易重复种子模式或遗漏边界情形;应与真实分布和难例对照,数量增长不代表覆盖增长。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • · 查看原书依据

      Data augmentation In this technique, the teacher is shown a set of seed input-output examples. Based on the seed examples, the teacher generates similar input-output examples, constituting the distillation dataset. Note that both the input and output are generated by the teacher model in this setting. The limitation of this technique is that the teacher is unable to generate sufficiently diverse examples. Intermediate representations This

    参考来源

    - 位置:《Designing Large Language Model Applications》第09章《推理优化》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Designing Large Language Model Applications/_epub-src/OEBPS/ch09.html) - 结论依据:摘录中关于蒸馏扩增要检查输入多样性的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。 - 原始内容:Data augmentation In this technique, the teacher is shown a set of seed input-output examples. Based on the seed examples, the teacher generates similar input-output examples, constituting the distillation dataset. Note that both the input and output are generated by the teacher model in this setting. The limitation of this technique is that the teacher is unable to generate sufficiently diverse examples. Intermediate representations This