知识卡片
教师响应构成任务蒸馏信号
用真实任务提示教师生成回答或过程,再把这些输出用于学生监督。要筛查错误、重复和任务覆盖;合成解释并非天然正确,不能因来自更大模型就省略质量验证。
这张卡片看不懂?
这张卡片已通过人工审查,不再接受失败反馈。
已有反馈 · 0 条
还没有人提交反馈。
在整书笔记中的位置
- · 推理优化
原书出处 · 1 处
- · 查看原书依据
Unsupervised generation In this technique, the teacher is prompted with instructions and/or examples for solving a task. The teacher’s responses comprise the distillation dataset. This technique is commonly used to teach capabilities like CoT or instruction-following to smaller models. To accomplish that, teacher models are asked to respond to queries with the thought process leading up to the answer. Data augmentation In this technique, the
参考来源
- 位置:《Designing Large Language Model Applications》第09章《推理优化》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Designing Large Language Model Applications/_epub-src/OEBPS/ch09.html)
- 结论依据:摘录中关于教师响应构成任务蒸馏信号的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。
- 原始内容:Unsupervised generation In this technique, the teacher is prompted with instructions and/or examples for solving a task. The teacher’s responses comprise the distillation dataset. This technique is commonly used to teach capabilities like CoT or instruction-following to smaller models. To accomplish that, teacher models are asked to respond to queries with the thought process leading up to the answer. Data augmentation In this technique, the