知识卡片
扩词表还需扩嵌入并学习
添加领域token后,模型的输入嵌入及相关输出权重维度须保持一致,并通过适当训练学习新项。仅更新tokenizer并resize不能赋予新词真实语义;还要检查新切分对旧输入和模板的影响。
这张卡片看不懂?
这张卡片已通过人工审查,不再接受失败反馈。
已有反馈 · 0 条
还没有人提交反馈。
在整书笔记中的位置
- · 词表与分词
原书出处 · 1 处
- · 查看原书依据
only halfway done here. The embedding vectors corresponding to these new tokens do not contain any information about these tokens. We will need to learn the right representations for these tokens, which we can do using fine-tuning or continued pre-training, which we will discuss in Chapter 7.
参考来源
- 位置:《Designing Large Language Model Applications》第03章《词表与分词》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Designing Large Language Model Applications/_epub-src/OEBPS/ch03.html)
- 结论依据:摘录中关于扩词表还需扩嵌入并学习的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。
- 原始内容:only halfway done here. The embedding vectors corresponding to these new tokens do not contain any information about these tokens. We will need to learn the right representations for these tokens, which we can do using fine-tuning or continued pre-training, which we will discuss in Chapter 7.