知识卡片
预处理有损与否要按任务比较
词干、词形还原和停用词规则会压缩变体,也可能丢失神经模型可利用的信号。可用保持其他条件一致的实验比较保留与删除,而非一律禁止;使用预训练模型还应遵循它训练时的分词和归一化契约。
这张卡片看不懂?
这张卡片已通过人工审查,不再接受失败反馈。
已有反馈 · 0 条
还没有人提交反馈。
在整书笔记中的位置
- · 神经网络基础
原书出处 · 1 处
- · 查看原书依据
Warning Avoid using stemmers, lemmatizers, or other keyword-based preprocessing in your deep learning pipeline unless you’re absolutely sure it is helping your model perform better for your application. You’ll find out that stemming and lemmatization “robs” your model of important information and decreases the amount of patterns and features it can find to do its job better.
参考来源
- 位置:《Natural Language Processing in Action, Second Edition》第05章《神经网络基础》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Natural Language Processing in Action, Second Edition/_epub-src/OEBPS/Text/chapter-5.html)
- 结论依据:摘录中关于预处理有损与否要按任务比较的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。
- 原始内容:Warning Avoid using stemmers, lemmatizers, or other keyword-based preprocessing in your deep learning pipeline unless you’re absolutely sure it is helping your model perform better for your application. You’ll find out that stemming and lemmatization “robs” your model of important information and decreases the amount of patterns and features it can find to do its job better.