知识卡片

普通读书笔记卡 · 6601.f · 生成模型:GPT-6

词表编号必须与嵌入语义对应

理解等级:高级

预训练tokenizer输出的ID须对应模型预训练嵌入和输出词表的同一符号。BART与GPT-2即使存在相近字节词表,也不能不检查编号、特殊token和偏移就互换;需要兼容tokenizer或明确重映射并相应调整模型。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • · 查看原书依据

      Let’s run the training procedure. First, we’ll set up our imports and our tokenizer. We’ll use Bart as the tokenizer for all of our models. Listing 13.7 Training the causal LLM import torch from sklearn.model_selection import train_test_split from torch.utils.data import Dataset from transformers import ( AutoModelForCausalLM, AutoModelForSeq2SeqLM, AutoTokenizer, DataCollatorForLanguageModeling, Seq2SeqTrainer, Seq2SeqTrainingArguments, Trainer, TrainingArguments) url = ("https://raw.githubusercontent.com/altdeep/" "causalML/master/book/chapter%2013/" "king-prince-kingdom-updated.csv") df = pd.read_csv(url) tokenizer = AutoTokenizer.from_pretrained("facebook/bart-base") #1

    参考来源

    - 位置:《Causal AI》第13章《因果与大语言模型》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Causal AI/_epub-src/OEBPS/Text/chapter-13.xhtml) - 结论依据:摘录中关于词表编号必须与嵌入语义对应的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。 - 原始内容:Let’s run the training procedure. First, we’ll set up our imports and our tokenizer. We’ll use Bart as the tokenizer for all of our models. Listing 13.7 Training the causal LLM import torch from sklearn.model_selection import train_test_split from torch.utils.data import Dataset from transformers import ( AutoModelForCausalLM, AutoModelForSeq2SeqLM, AutoTokenizer, DataCollatorForLanguageModeling, Seq2SeqTrainer, Seq2SeqTrainingArguments, Trainer, TrainingArguments) url = ("https://raw.githubusercontent.com/altdeep/" "causalML/master/book/chapter%2013/" "king-prince-kingdom-updated.csv") df = pd.read_csv(url) tokenizer = AutoTokenizer.from_pretrained("facebook/bart-base") #1