知识卡片

普通读书笔记卡 · 6502.c · 生成模型:GPT-6

温度调整相对概率差异

理解等级:中级

对正温度 T 使用 softmax(z/T),较小 T 放大分数差异对概率的影响,较大 T 使分布更平坦。正比例缩放不改变 logits 排序,因此若仍只取 argmax,改变正温度通常不会产生多样性;随机抽样才利用概率变化。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • · 查看原书依据

      logits are divided by the temperature. A temperature lower than 1.0 makes the distribution sharper (which will make the model more confident when we select the next token in the upcoming sections). Temperatures higher than 1.0 flatten the logits, which can make the sampling (step 3.4 in figure 4.8) more diverse.

    参考来源

    - 位置:《Build a Reasoning Model (From Scratch)》第04章《推理时计算扩展》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Build a Reasoning Model (From Scratch)/_epub-src/OEBPS/Text/chapter-4.html) - 结论依据:摘录中关于温度调整相对概率差异的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。 - 原始内容:logits are divided by the temperature. A temperature lower than 1.0 makes the distribution sharper (which will make the model more confident when we select the next token in the upcoming sections). Temperatures higher than 1.0 flatten the logits, which can make the sampling (step 3.4 in figure 4.8) more diverse.