知识卡片

普通读书笔记卡 · 6719.b · 生成模型:GPT-6

去词与未知词处理会改变相邻关系

理解等级:中级

删除词项使原本隔开的词变成相邻,影响卷积核捕捉的模式。URL、坐标和高频片段是否删除应依据任务,有时本身就是信息;应保存训练拟合的词表及规则,测试只复用,而非看全数据再挑词。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • · 查看原书依据

      You can see in these examples that some microblog posts push right up against the character limit of Birdsite. Others get the point across with fewer words. So you will need to pad, or fill, these shorter texts so all of the examples in your dataset have the same number of tokens. If you plan to filter out very frequently or rarely used words later

    参考来源

    - 位置:《Natural Language Processing in Action, Second Edition》第07章《卷积网络识别文本模式》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Natural Language Processing in Action, Second Edition/_epub-src/OEBPS/Text/chapter-7.html) - 结论依据:摘录中关于去词与未知词处理会改变相邻关系的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。 - 原始内容:You can see in these examples that some microblog posts push right up against the character limit of Birdsite. Others get the point across with fewer words. So you will need to pad, or fill, these shorter texts so all of the examples in your dataset have the same number of tokens. If you plan to filter out very frequently or rarely used words later