知识卡片

普通读书笔记卡 · 6701.b · 生成模型:GPT-6

训练损失接口必须匹配输出形式

理解等级:中级

CrossEntropyLoss接收原始logits与相应目标编号,可内部完成所需归一计算。若提前把logits当概率或错误地重复softmax,将改变数值与训练行为;同一批次输入、目标及设备必须保持一致。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • · 查看原书依据

      loss_fn = nn.CrossEntropyLoss() optimizer = torch.optim.AdamW(model.parameters(), lr=LEARNING_RATE)running_loss = [] pbar = tqdm(range(EPOCHS * len(dataloader['train']))) for epoch in range(EPOCHS): epoch_loss = 0 for sample_num, (center, context) in enumerate(dataloader['train']): if sample_num % len(dataloader['train']) == 2: print(center, context) # center: tensor([ 229, 0, 2379, ..., 402, 553, 521]) # context: tensor([ 112, 1734, 802, ..., 28, 852, 363]) center, context = center.to(device), context.to(device) optimizer.zero_grad() logits = model(input=context) loss

    参考来源

    - 位置:《Natural Language Processing in Action, Second Edition》第06章《用词嵌入推理》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Natural Language Processing in Action, Second Edition/_epub-src/OEBPS/Text/chapter-6.html) - 结论依据:摘录中关于训练损失接口必须匹配输出形式的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。 - 原始内容:loss_fn = nn.CrossEntropyLoss() optimizer = torch.optim.AdamW(model.parameters(), lr=LEARNING_RATE)running_loss = [] pbar = tqdm(range(EPOCHS * len(dataloader['train']))) for epoch in range(EPOCHS): epoch_loss = 0 for sample_num, (center, context) in enumerate(dataloader['train']): if sample_num % len(dataloader['train']) == 2: print(center, context) # center: tensor([ 229, 0, 2379, ..., 402, 553, 521]) # context: tensor([ 112, 1734, 802, ..., 28, 852, 363]) center, context = center.to(device), context.to(device) optimizer.zero_grad() logits = model(input=context) loss