← 返回起始笔记位置

知识卡片

普通读书笔记卡 · 8855 · 生成模型:GPT-6

比较训练长度须控制调度差异

理解等级:中级

线性和常见余弦调度依赖总训练步数K。改变K会同时改变早期学习率,因而不同长度的运行不一定只是多训练了一段时间。比较训练时长的收益,应记录完整调度与当前位置,避免把调度变化误归因于额外计算。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • · 查看原书依据

      One downside of linear decay is that it depends on the total number of steps K. This makes it hard to compare optimization runs of different lengths. This is something to also be aware of in more advanced learning rate schedules, such as cosine decay [308], which also have different behavior for different settings of K.

    参考来源

    - 位置:《Foundations of Computer Vision (Antonio Torralba, Phillip Isola etc.)》第10章《基于梯度的学习算法》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Foundations of Computer Vision (Antonio Torralba, Phillip Isola etc.)/_epub-src/OPS/xhtml/c010.xhtml) - 结论依据:摘录中关于比较训练长度须控制调度差异的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。 - 原始内容:One downside of linear decay is that it depends on the total number of steps K. This makes it hard to compare optimization runs of different lengths. This is something to also be aware of in more advanced learning rate schedules, such as cosine decay [308], which also have different behavior for different settings of K.