原书出处
Build a Reasoning Model (From Scratch)
target tokens are simply the input tokens shifted by one position, since an LLM is trained to predict the next token in a sequence. For example, given the input "The capital of Germany is Berlin", the model is asked at position 2 (the token for "capital") to predict position 3 (the token for "of"). Using all_probas[t_idx, next_ids] (steps 4 and 5) retrieves exactly these probability
原书定位详情
来源:a19c7933-4eda-4176-aebd-30fe7e1ad292
正文位置:extracted/Build a Reasoning Model (From Scratch)/_epub-src/OEBPS/Text/chapter-5.html
EPUB SHA-256:52d4d07986020be70f5aac21762dfcf60fcba08db0ba29d4f63957caf12aa377