知识卡片

普通读书笔记卡 · 6665.c · 生成模型:GPT-6

拟合词表时区分检索与预测

理解等级:中级

检索可对当前被索引文档拟合词表及DF,查询复用这一变换。监督预测的独立测试则不应参与训练用词表、阈值或IDF拟合,以免测试分布影响训练表示;两种任务的数据边界不同,不能一概把全索引拟合叫泄漏。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • · 查看原书依据

      It is usually best to run through your entire corpus to build up your vocabulary before you go back through the documents, counting up tokens and putting them in the right vocabulary slot. If you do it this way, you can alphabetize your vocabulary, making it easier to keep track of approximately where each token count should be in the vectors. This approach also allows

    参考来源

    - 位置:《Natural Language Processing in Action, Second Edition》第03章《词频与逆文档频率向量》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Natural Language Processing in Action, Second Edition/_epub-src/OEBPS/Text/chapter-3.html) - 结论依据:摘录中关于拟合词表时区分检索与预测的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。 - 原始内容:It is usually best to run through your entire corpus to build up your vocabulary before you go back through the documents, counting up tokens and putting them in the right vocabulary slot. If you do it this way, you can alphabetize your vocabulary, making it easier to keep track of approximately where each token count should be in the vectors. This approach also allows