知识卡片

普通读书笔记卡 · 8703.h · 生成模型:GPT-6

后分块先全局编码再局部池化

理解等级:高级

将长文在模型支持范围内先编码成上下文化token,再按段界池化成多个向量,使局部表示可能携带远处背景。它受有效上下文、编码方向与模型训练限制,不是把任意超长文无损压入每个块。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • · 查看原书依据

      Late Chunking One way of supporting long-range dependencies in text is to use late chunking, a method introduced by Jina AI. Recall from earlier in the chapter that embeddings are generated by typically pooling the vectors from the last layer of the underlying language model. Given that we have access to long-context language models that can accept an entire long document in a single input,

    参考来源

    - 位置:《Designing Large Language Model Applications》第11章《表示学习与嵌入》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Designing Large Language Model Applications/_epub-src/OEBPS/ch11.html) - 结论依据:摘录中关于后分块先全局编码再局部池化的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。 - 原始内容:Late Chunking One way of supporting long-range dependencies in text is to use late chunking, a method introduced by Jina AI. Recall from earlier in the chapter that embeddings are generated by typically pooling the vectors from the last layer of the underlying language model. Given that we have access to long-context language models that can accept an entire long document in a single input,