知识卡片
序列上限应权衡截断与资源
提高max_seq_length可保留更多评论,但通常增加计算及激活存储,需配合批量大小。上限由模型配置和实际实现共同约束,最优长度需看文本分布与验证,不能以显存刚好够就定为最大。
这张卡片看不懂?
这张卡片已通过人工审查,不再接受失败反馈。
已有反馈 · 0 条
还没有人提交反馈。
在整书笔记中的位置
原书出处 · 1 处
- · 查看原书依据
With the text cleaned, turn your focus to tuning the model initialization and training parameters. In your first training run, you accepted the default input sequence length (128), as an explicit value for max_sequence_length was not provided to the model. The BERT base model can handle sequences of a maximum length of 512. As you increase max_sequence_length, you may need to decrease train_batch_size and eval_batch_size
参考来源
- 位置:《Natural Language Processing in Action, Second Edition》第09章《可堆叠的Transformer》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Natural Language Processing in Action, Second Edition/_epub-src/OEBPS/Text/chapter-9.html)
- 结论依据:摘录中关于序列上限应权衡截断与资源的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。
- 原始内容:With the text cleaned, turn your focus to tuning the model initialization and training parameters. In your first training run, you accepted the default input sequence length (128), as an explicit value for max_sequence_length was not provided to the model. The BERT base model can handle sequences of a maximum length of 512. As you increase max_sequence_length, you may need to decrease train_batch_size and eval_batch_size