← 返回刚才阅读的位置

知识卡片

普通读书笔记卡 · 6507.c · 生成模型:GPT-6

广播维度影响批量归一化

理解等级:高级

对 [B,V] 概率逐行求和后应保留 [B,1] 分母以明确沿词表广播。本章 sum(dim=1) 得 [B],在 B=1 可工作,却不能泛化为任意批量。单样本成功不等于张量接口具备批处理语义。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • · 查看原书依据

      denom = torch.sum(filtered).clamp_min(1e-12) renormalized = filtered / denom print(renormalized) The resulting, normalized tensor is: tensor([0.0000, 0.0000, 0.0000, 0.0000, 0.6225, 0.0000, 0.0000, 0.0000, 0.0000, 0.3775]) The goal of top-p filtering, which we implemented in this section, is to remove tokens with low probabilities to avoid them from being sampled later. This helps reduce nonsensical token responses in given contexts. 4.5.2 Adding a top-p filter to the

    参考来源

    - 位置:《Build a Reasoning Model (From Scratch)》第04章《推理时计算扩展》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Build a Reasoning Model (From Scratch)/_epub-src/OEBPS/Text/chapter-4.html) - 结论依据:摘录中关于广播维度影响批量归一化的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。 - 原始内容:denom = torch.sum(filtered).clamp_min(1e-12) renormalized = filtered / denom print(renormalized) The resulting, normalized tensor is: tensor([0.0000, 0.0000, 0.0000, 0.0000, 0.6225, 0.0000, 0.0000, 0.0000, 0.0000, 0.3775]) The goal of top-p filtering, which we implemented in this section, is to remove tokens with low probabilities to avoid them from being sampled later. This helps reduce nonsensical token responses in given contexts. 4.5.2 Adding a top-p filter to the