知识卡片

普通读书笔记卡 · 6671.c · 生成模型:GPT-6

长度归一与词频饱和可以分开

理解等级:中级

BM25一类排序在跨文档稀有度之外,考虑单篇词频的边际收益递减,以及文档相对平均长度的影响。它不是简单对最终余弦分数取平均;书中缩写公式不能替代完整逐查询词求和的标准实现,参数须在检索评价上比较。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • · 查看原书依据

      TF–IDF matrices (term-document matrices) have been the mainstay of information retrieval (search) for decades. As a result, researchers and corporations have spent a lot of time trying to optimize that IDF part to try to improve the relevance of search results. One such alternative to using TF–IDF vectors is called Okapi BM25, or its most recent variant, BM25F.38

    参考来源

    - 位置:《Natural Language Processing in Action, Second Edition》第03章《词频与逆文档频率向量》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Natural Language Processing in Action, Second Edition/_epub-src/OEBPS/Text/chapter-3.html) - 结论依据:摘录中关于长度归一与词频饱和可以分开的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。 - 原始内容:TF–IDF matrices (term-document matrices) have been the mainstay of information retrieval (search) for decades. As a result, researchers and corporations have spent a lot of time trying to optimize that IDF part to try to improve the relevance of search results. One such alternative to using TF–IDF vectors is called Okapi BM25, or its most recent variant, BM25F.38