知识卡片

普通读书笔记卡 · 7005.l · 生成模型:DeepSeek-V4.1-Flash

深度学习的特殊脆弱性

理解等级:高级

深度学习预测性能极强,却带一组特有的脆弱性。它从随机初始态出发,把海量数据按随机顺序、小批量喂入,每步算损失梯度后微调内部,这种「随机梯度下降 SGD」要重复百万到十亿次;随机化贯穿初始态、数据顺序与并行更新,同配置同数据重训也可能得到差别很大的模型。由此引出:难判断训练何时「该停」;步子稍大就梯度爆炸、吐 NaN;对学习率等超参极敏感;算力可达数千 CPU 核连跑数周;且越陌生的样本越易外推出自信却离谱的极端误差。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • Basic Introduction to Models · 查看原书依据

      When deep learning methods are wrong, they might be very wrong Deep learning methods extrapolate from their training data, which means that the more unfamiliar a new previously unseen data point is, the more likely we are to have an extreme prediction that might be completely off base or out of the range of typical behavior. These sorts of highly confident errors can be a significant source of system-level misbehavior.

    参考来源

    - 位置:《Reliable Machine Learning》第03章《Basic Introduction to Models》(源文件:extracted/Reliable Machine Learning/_epub-src/OEBPS/ch03.xhtml) - 结论依据:原文明确阐述该主题,并给出定义、实践或反例。 - 原始内容:When deep learning methods are wrong, they might be very wrong Deep learning methods extrapolate from their training data, which means that the more unfamiliar a new previously unseen data point is, the more likely we are to have an extreme prediction that might be completely off base or out of the range of typical behavior. These sorts of highly confident errors can be a significant source of system-level misbehavior.