知识卡片
序列级早退减少整段计算深度
序列可固定在某层结束,或用判别模块决定退出层。中间层输出需能承担目标任务,不能直接假定删去后层行为不变;动态判别器的成本及错误退出风险应和节省的计算共同测量。
这张卡片看不懂?
这张卡片已通过人工审查,不再接受失败反馈。
已有反馈 · 0 条
还没有人提交反馈。
在整书笔记中的位置
- · 推理优化
原书出处 · 1 处
- · 查看原书依据
Sequence-level early exit In this scenario, the forward pass in the Transformer is stopped at a particular layer for the entire input sequence, and the intermediate representations from that layer are taken as the final output. The layer at which to exit can be determined in advance or can be dynamically decided based on the input sequence. To dynamically decide the layer to exit, you
参考来源
- 位置:《Designing Large Language Model Applications》第09章《推理优化》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Designing Large Language Model Applications/_epub-src/OEBPS/ch09.html)
- 结论依据:摘录中关于序列级早退减少整段计算深度的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。
- 原始内容:Sequence-level early exit In this scenario, the forward pass in the Transformer is stopped at a particular layer for the entire input sequence, and the intermediate representations from that layer are taken as the final output. The layer at which to exit can be determined in advance or can be dynamically decided based on the input sequence. To dynamically decide the layer to exit, you