知识卡片

普通读书笔记卡 · 6694.c · 生成模型:GPT-6

动量积累近期梯度方向

理解等级:中级

动量将过去更新或梯度信息与当前梯度结合,在连续一致方向上可加快推进,也可能在弯曲区域造成过冲。它不观察真实全局最小点,不能把加速与减速拟人化为已知正确方向;参数需按验证与稳定性调节。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • · 查看原书依据

      Momentum is an attribute of our gradient descent algorithm that allows it to “accelerate” when it’s moving in the right direction and “slow down” when it’s getting away from its target. How do we decide which values to give these two attributes? As with other hyperparameters you see in this book, you’ll need to optimize them to see what’s the most effective one for your

    参考来源

    - 位置:《Natural Language Processing in Action, Second Edition》第05章《神经网络基础》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Natural Language Processing in Action, Second Edition/_epub-src/OEBPS/Text/chapter-5.html) - 结论依据:摘录中关于动量积累近期梯度方向的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。 - 原始内容:Momentum is an attribute of our gradient descent algorithm that allows it to “accelerate” when it’s moving in the right direction and “slow down” when it’s getting away from its target. How do we decide which values to give these two attributes? As with other hyperparameters you see in this book, you’ll need to optimize them to see what’s the most effective one for your