知识卡片
训练保留软输出再作离散选择
Argmax提供类别索引却几乎处处没有有用梯度,分类训练通常保留可求导的分数与损失,再在需要决策时选择类别。目标是提高正确类别的分配,而不是要求每一隐藏层都输出独热向量或每处都处处光滑。
这张卡片看不懂?
这张卡片已通过人工审查,不再接受失败反馈。
已有反馈 · 0 条
还没有人提交反馈。
在整书笔记中的位置
- · 人工神经网络
原书出处 · 1 处
- · 查看原书依据
So, why not just normalize with the actual values—that is, using zj instead of ezj or, even easier, using arg maxj to set the highest value of z to 1 and leave the rest at 0? The reason is that each layer needs to remain differentiable for backpropagation to work. Yet the “brutal” cutoff introduced by the arg max function is exactly what we want
参考来源
- 位置:《Introduction to Autonomous Robots Mechanisms, Sensors, Actuators, and Algorithms (Nikolaus Correll, Bradley Hayes etc.)》第10章《人工神经网络》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Introduction to Autonomous Robots Mechanisms, Sensors, Actuators, and Algorithms (Nikolaus Correll, Bradley Hayes etc.)/_epub-src/OEBPS/xhtml/chapter_10.xhtml)
- 结论依据:摘录中关于训练保留软输出再作离散选择的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。
- 原始内容:So, why not just normalize with the actual values—that is, using zj instead of ezj or, even easier, using arg maxj to set the highest value of z to 1 and leave the rest at 0? The reason is that each layer needs to remain differentiable for backpropagation to work. Yet the “brutal” cutoff introduced by the arg max function is exactly what we want