知识卡片

普通读书笔记卡 · 6656.e · 生成模型:GPT-6

两种子词合并算法不同

理解等级:中级

两者都可学习子词,但合并选择及实际切分流程不同,不能将WordPiece说成边训练底层大语言模型边删除字符。Hugging Face教程给出的训练重建带明确推测声明,理解时应区分公开实现事实和教学重建。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • · 查看原书依据

      Note You may see mention of WordPiece tokenizers, such as BERT and its derivatives, which are used in some advanced language models.15 This process works the same as BPE, but it actually uses the underlying language model to predict the neighboring characters in a string. It eliminates the characters from its vocabulary that lower the accuracy of this language model the least. The math is

    参考来源

    - 位置:《Natural Language Processing in Action, Second Edition》第02章《思维的词元》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Natural Language Processing in Action, Second Edition/_epub-src/OEBPS/Text/chapter-2.html) - 结论依据:摘录中关于两种子词合并算法不同的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。 - 原始内容:Note You may see mention of WordPiece tokenizers, such as BERT and its derivatives, which are used in some advanced language models.15 This process works the same as BPE, but it actually uses the underlying language model to predict the neighboring characters in a string. It eliminates the characters from its vocabulary that lower the accuracy of this language model the least. The math is