知识卡片
范数球约束限制模型权重
以∥w∥≤B约束线性模型权重,可限制权重整体规模。
投影到该球时,
- 候选w′已在球内就保持不变;
- 超出球时才缩放为(B/∥w′∥)w′。
书中直接缩放公式须加上这个条件。
这张卡片看不懂?
这张卡片已通过人工审查,不再接受失败反馈。
已有反馈 · 0 条
还没有人提交反馈。
在整书笔记中的位置
原书出处 · 1 处
- 第33章 机器学习算法 · 查看原书依据
One way to regularize this example would be to explicitly limit the norm of the weights, adding a constraint that ∥w∥ ≤ B for some bound B > 0. (Recall again that the components of the vector w are the variables in the present application.) Adding this constraint controls the complexity of the model, as the number of values wj that can have large absolute value is now limited. In order to run GRADIENT-DESCENT-CONSTRAINED for any problem, you need to implement the projection step, as well as to compute bounds on R and L. We conclude this section by describing these calculations for gradient descent with the constraint ∥w∥ ≤ B. First, consider the projection step in line 5. Suppose that the update in line 4 results in a vector w′. The projection is implemented by computing ∏k(w′) where K is defined by ∥w∥ ≤ B. This