知识卡片

普通读书笔记卡 · 6601.d · 生成模型:GPT-6

对数概率相加仍是对数概率

理解等级:中级

自回归序列概率是逐步条件概率的乘积,对数概率则相加。若累加得到log P,需要指数转换才是P;同时应明确是否计入EOS、是否只计续写、以及padding是否排除,不能把不同评分范围直接比较。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • · 查看原书依据

      Summing these together provides the joint probability of the sequence under the model. Of course, as a generative model, GPT-2 can generate the next token conditional on the tokens that came before it. The prompt the user provides is the beginning of the sequence, and the model’s response extends the sequence. Listing 13.5 Generation from the LLM prompt = "Counterfactual reasoning would enable AI to"

    参考来源

    - 位置:《Causal AI》第13章《因果与大语言模型》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Causal AI/_epub-src/OEBPS/Text/chapter-13.xhtml) - 结论依据:摘录中关于对数概率相加仍是对数概率的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。 - 原始内容:Summing these together provides the joint probability of the sequence under the model. Of course, as a generative model, GPT-2 can generate the next token conditional on the tokens that came before it. The prompt the user provides is the beginning of the sequence, and the model’s response extends the sequence. Listing 13.5 Generation from the LLM prompt = "Counterfactual reasoning would enable AI to"