返回主题:线性模型与优化算法

知识卡片

普通读书笔记卡 · 3710.h.2

范数球约束限制模型权重

理解等级:中级

以∥w∥≤B约束线性模型权重,可限制权重整体规模。

投影到该球时,

  • 候选w′已在球内就保持不变;
  • 超出球时才缩放为(B/∥w′∥)w′。

书中直接缩放公式须加上这个条件。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • 第33章 机器学习算法 · 查看原书依据

      One way to regularize this example would be to explicitly limit the norm of the weights, adding a constraint that ∥w∥ ≤ B for some bound B > 0. (Recall again that the components of the vector w are the variables in the present application.) Adding this constraint controls the complexity of the model, as the number of values wj that can have large absolute value is now limited. In order to run GRADIENT-DESCENT-CONSTRAINED for any problem, you need to implement the projection step, as well as to compute bounds on R and L. We conclude this section by describing these calculations for gradient descent with the constraint ∥w∥ ≤ B. First, consider the projection step in line 5. Suppose that the update in line 4 results in a vector w′. The projection is implemented by computing ∏k(w′) where K is defined by ∥w∥ ≤ B. This

    参考来源

    - 位置:《算法导论(第4版)》第33章《机器学习算法》(OEBPS/chapter033.xhtml,文本字符[65000,70000))。 - 结论依据:原文第33.3节或章末题目、注记;相关公式图已核对,未解答书中练习。 - 原始内容:One way to regularize this example would be to explicitly limit the norm of the weights, adding a constraint that ∥ w ∥ ≤ B for some bound B > 0. (Recall again that the components of the vector w are the variables in the present application.) Adding this constraint controls the complexity of the model, as the number of values w j that can have large absolute value is now limited. In order to run G RADIENT -D ESCENT -C ONSTRAINED for any problem, you need to implement the projection step, as well as to compute bounds on R and L . We conclude this section by describing these calculations for gradient descent with the constraint ∥ w ∥ ≤ B . First, consider the projection step in line 5. Suppose that the update in line 4 results in a vector w ′. The projection is implemented by computing ∏ k ( w ′) where K is defined by ∥ w ∥ ≤ B . This