知识卡片

普通读书笔记卡 · 6763.d · 生成模型:GPT-6

整理标签不负责改变模型可见性

理解等级:中级

mlm=False让语言建模collator按因果训练形式准备标签并忽略padding。目标右移与因果注意由模型实现相应处理;collator本身不把任意双向模型改造成自回归模型,也不直接规定输入注意力矩阵。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • · 查看原书依据

      The mlm=False (masked language model) setting is an especially tricky quirk of transformers. This is your way of declaring that the dataset used for training your model need only be given the tokens in the causal direction—left to right, in English. You would need to set this to True if you were feeding the trainer a dataset that had random tokens masked. This is the

    参考来源

    - 位置:《Natural Language Processing in Action, Second Edition》第10章《真实世界中的大语言模型》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Natural Language Processing in Action, Second Edition/_epub-src/OEBPS/Text/chapter-10.html) - 结论依据:摘录中关于整理标签不负责改变模型可见性的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。 - 原始内容:The mlm=False (masked language model) setting is an especially tricky quirk of transformers. This is your way of declaring that the dataset used for training your model need only be given the tokens in the causal direction—left to right, in English. You would need to set this to True if you were feeding the trainer a dataset that had random tokens masked. This is the