← 返回刚才阅读的位置

知识卡片

普通读书笔记卡 · 9094.a · 生成模型:GPT-6

分词在词表与长度之间取舍

理解等级:中级

整词词表难覆盖新名字和程序变量,字符编码较易覆盖却增加序列长度,子词方案在两者间折中。字节对编码的实际覆盖取决于基础符号与回退规则,少数英文字符不能代表全部语言;语义单位也不必与一个token一致。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • · 查看原书依据

      A happy middle ground can be achieved using byte-pair encodings [150]. In this approach, we assign a one-hot code to each small chunk of text. The chunks are common substrings found in text, like “th” or “ing.” This strategy is popular in language models like the generative pretrained transformer (GPT) series [63] and the contrastive language-image pretraining (CLIP) model [394] that we will learn about

    参考来源

    - 位置:《Foundations of Computer Vision (Antonio Torralba, Phillip Isola etc.)》第51章《视觉与语言》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Foundations of Computer Vision (Antonio Torralba, Phillip Isola etc.)/_epub-src/OPS/xhtml/c051.xhtml) - 结论依据:摘录中关于分词在词表与长度之间取舍的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。 - 原始内容:A happy middle ground can be achieved using byte-pair encodings [150]. In this approach, we assign a one-hot code to each small chunk of text. The chunks are common substrings found in text, like “th” or “ing.” This strategy is popular in language models like the generative pretrained transformer (GPT) series [63] and the contrastive language-image pretraining (CLIP) model [394] that we will learn about