知识卡片

普通读书笔记卡 · 8854.b · 生成模型:GPT-6

学习率与步数是不同控制量

理解等级:中级

学习率控制每次梯度更新的尺度,步数控制执行多久。更大学习率可能更快,也可能越过低点甚至发散;更多步数也不能修复失效方向。原书关于小步长与随机初始化的局部收敛结论依赖目标光滑性等条件,不能泛化为所有损失。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • · 查看原书依据

      This algorithm has two hyperparameters, the learning rate η, which controls the step size (learning rate times gradient magnitude), and the number of steps K. If the learning rate is sufficiently small and the initial parameter vector θ0 is random, then this algorithm will almost surely converge to a local minimum of J as K → ∞ [288]. However, to descend more quickly, it can

    参考来源

    - 位置:《Foundations of Computer Vision (Antonio Torralba, Phillip Isola etc.)》第10章《基于梯度的学习算法》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Foundations of Computer Vision (Antonio Torralba, Phillip Isola etc.)/_epub-src/OPS/xhtml/c010.xhtml) - 结论依据:摘录中关于学习率与步数是不同控制量的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。 - 原始内容:This algorithm has two hyperparameters, the learning rate η, which controls the step size (learning rate times gradient magnitude), and the number of steps K. If the learning rate is sufficiently small and the initial parameter vector θ0 is random, then this algorithm will almost surely converge to a local minimum of J as K → ∞ [288]. However, to descend more quickly, it can