← 返回刚才阅读的位置

知识卡片

普通读书笔记卡 · 6657.a · 生成模型:GPT-6

增大词组范围会扩展稀疏特征空间

理解等级:中级

更长词组保留更多局部关系,同时组合数量增加、出现更稀疏,覆盖它们需更多样本或选择机制。书中20k与百万等规模属于示例统计,不是所有语料的固定阈值;应按验证表现、内存和覆盖共同选择范围。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • · 查看原书依据

      Designing a filter for stop words depends on your particular application. Vocabulary size will drive the computational complexity and memory requirements of all subsequent steps in the NLP pipeline, but stop words are only a small portion of your total vocabulary size. A typical stop word list has only 100 or so frequent and unimportant words listed in it. On the other hand, a vocabulary

    参考来源

    - 位置:《Natural Language Processing in Action, Second Edition》第02章《思维的词元》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Natural Language Processing in Action, Second Edition/_epub-src/OEBPS/Text/chapter-2.html) - 结论依据:摘录中关于增大词组范围会扩展稀疏特征空间的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。 - 原始内容:Designing a filter for stop words depends on your particular application. Vocabulary size will drive the computational complexity and memory requirements of all subsequent steps in the NLP pipeline, but stop words are only a small portion of your total vocabulary size. A typical stop word list has only 100 or so frequent and unimportant words listed in it. On the other hand, a vocabulary