← 返回刚才阅读的位置

知识卡片

普通读书笔记卡 · 8857.c · 生成模型:GPT-6

随机梯度的噪声既可助也可扰

理解等级:中级

不同批次产生不同更新方向,有时能越过全量梯度下的小障碍,也可能引入持续波动。隐式正则化描述算法偏好某些解,不等于对任意模型都保证泛化;线性情形的最小范数倾向还依赖初始化、损失与收敛等条件。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • · 查看原书依据

      SGD can jump over small bumps in the loss landscape, as long those bumps disappear for some randomly sampled batches. Another important property is that SGD can implicitly regularize the learning problem. For example, for linear problems (i.e., fθ is linear), then if there are multiple parameter settings that minimize the loss, SGD will often converge to the solution with minimum parameter norm [517].

    参考来源

    - 位置:《Foundations of Computer Vision (Antonio Torralba, Phillip Isola etc.)》第10章《基于梯度的学习算法》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Foundations of Computer Vision (Antonio Torralba, Phillip Isola etc.)/_epub-src/OPS/xhtml/c010.xhtml) - 结论依据:摘录中关于随机梯度的噪声既可助也可扰的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。 - 原始内容:SGD can jump over small bumps in the loss landscape, as long those bumps disappear for some randomly sampled batches. Another important property is that SGD can implicitly regularize the learning problem. For example, for linear problems (i.e., fθ is linear), then if there are multiple parameter settings that minimize the loss, SGD will often converge to the solution with minimum parameter norm [517].