← 返回卡片

原书出处

Designing Large Language Model Applications

First Edition · 核验版本的摘录快照 · 导入于 2026-11-14T13:54:13.933Z

Attention with Linear Biases (ALiBi) In this technique, the attention scores are penalized with a bias term proportional to the distance between the query token and the key token. This technique also enables modeling sequences of longer length during inference than what was encountered in the training process. Rotary Position Embedding (RoPE) Just like ALiBi, this technique has the property of relative decay; there is

此预览只包含定位所需的摘录和上下文。

原书定位详情

来源:a19c7933-4eda-4176-aebd-30fe7e1ad292

正文位置:extracted/Designing Large Language Model Applications/_epub-src/OEBPS/ch04.html

EPUB SHA-256:1b4d2347fec2f949fcddfbaf4fad597a1ae20887a096cfae64c59a619b4781bf