← 返回刚才阅读的位置

知识卡片

普通读书笔记卡 · 8867.a · 生成模型:GPT-6

归一化层可能耦合批内预测

理解等级:中级

当某样本输出依赖当前批均值和方差时,更换同批其他样本也可能改变该样本表示。依赖逐样本独立计算的推导需检查这一条件。使用固定累计统计的推理阶段又不同,不能把训练时的批耦合无条件套到所有推理配置。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • · 查看原书依据

      introduces a dependency between each datapoint in the batch. This violates the principle that datapoints should be processed independently and identically (iid), and this can lead to bugs if your method relies on the iid assumption. Layernorm does not have this problem and does indeed process each datapoint in an iid fashion.

    参考来源

    - 位置:《Foundations of Computer Vision (Antonio Torralba, Phillip Isola etc.)》第12章《神经网络》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Foundations of Computer Vision (Antonio Torralba, Phillip Isola etc.)/_epub-src/OPS/xhtml/c012.xhtml) - 结论依据:摘录中关于归一化层可能耦合批内预测的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。 - 原始内容:introduces a dependency between each datapoint in the batch. This violates the principle that datapoints should be processed independently and identically (iid), and this can lead to bugs if your method relies on the iid assumption. Layernorm does not have this problem and does indeed process each datapoint in an iid fashion.