← 返回刚才阅读的位置

知识卡片

普通读书笔记卡 · 8698.b · 生成模型:GPT-6

均值池化需要屏蔽填充位置

理解等级:中级

均值池化汇总有效token表示,padding不应作为普通文本参与平均。特殊token、归一化及截断也须与训练一致;简单平均所有张量位置可能让同句在不同批次长度下得到不同表示。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • · 查看原书依据

      Mean pooling, where the average is taken across all token outputs in the sequence Max pooling, where the maximum value in each dimension across all tokens is taken Weighted mean, where more weight is given to the last few tokens Last token, where the embedding is just the encoder output of the last token Tip Whether the last token (or the first token) contains good

    参考来源

    - 位置:《Designing Large Language Model Applications》第11章《表示学习与嵌入》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Designing Large Language Model Applications/_epub-src/OEBPS/ch11.html) - 结论依据:摘录中关于均值池化需要屏蔽填充位置的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。 - 原始内容:Mean pooling, where the average is taken across all token outputs in the sequence Max pooling, where the maximum value in each dimension across all tokens is taken Weighted mean, where more weight is given to the last few tokens Last token, where the embedding is just the encoder output of the last token Tip Whether the last token (or the first token) contains good