知识卡片

普通读书笔记卡 · 8670.c · 生成模型:GPT-6

删减适配层须重新测质量

理解等级:中级

减少或跳过适配器可降低部分推理成本,却改变训练得到的计算路径。应检验具体层删除后的目标质量与旧能力,不能直接把“激活小”当完全无用或保证丢掉后输出不变。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • · 查看原书依据

      AdapterDrop, a set of methods for dropping adapter modules during training and inference. They propose dropping adapters from the first few layers of the Transformer during inference or pruning the adapters from each layer that is the least

    参考来源

    - 位置:《Designing Large Language Model Applications》第07章《进阶微调技术》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Designing Large Language Model Applications/_epub-src/OEBPS/ch07.html) - 结论依据:摘录中关于删减适配层须重新测质量的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。 - 原始内容:AdapterDrop, a set of methods for dropping adapter modules during training and inference. They propose dropping adapters from the first few layers of the Transformer during inference or pruning the adapters from each layer that is the least