知识卡片

普通读书笔记卡 · 8702.b · 生成模型:GPT-6

层数与维度是不同压缩轴

理解等级:高级

可训练较早层输出可用句向量,再结合MRL裁剪维度。减层主要影响编码计算,减维主要影响向量存储和检索,二者质量变化不相同;层间KL需明确构成的分布,不能直接把任意隐藏向量当概率。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • · 查看原书依据

      The Sentence Transformers library allows you to combine Matryoshka representations with layer reduction using the Matryoshka2dLoss. Exercise Download this dataset by Rishabh Misra containing news headlines. Use the nomic-embed-text-v1.5 model from Nomic AI, which has been trained using MRL. Pick one of the headlines as the query and generate its query embedding. Generate document embeddings for all other headlines, and calculate similarity scores between query

    参考来源

    - 位置:《Designing Large Language Model Applications》第11章《表示学习与嵌入》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Designing Large Language Model Applications/_epub-src/OEBPS/ch11.html) - 结论依据:摘录中关于层数与维度是不同压缩轴的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。 - 原始内容:The Sentence Transformers library allows you to combine Matryoshka representations with layer reduction using the Matryoshka2dLoss. Exercise Download this dataset by Rishabh Misra containing news headlines. Use the nomic-embed-text-v1.5 model from Nomic AI, which has been trained using MRL. Pick one of the headlines as the query and generate its query embedding. Generate document embeddings for all other headlines, and calculate similarity scores between query