原书出处
Natural Language Processing in Action, Second Edition
Note You may see mention of WordPiece tokenizers, such as BERT and its derivatives, which are used in some advanced language models.15 This process works the same as BPE, but it actually uses the underlying language model to predict the neighboring characters in a string. It eliminates the characters from its vocabulary that lower the accuracy of this language model the least. The math is
原书定位详情
来源:a19c7933-4eda-4176-aebd-30fe7e1ad292
正文位置:extracted/Natural Language Processing in Action, Second Edition/_epub-src/OEBPS/Text/chapter-2.html
EPUB SHA-256:0e9a35e49ca446e322e5524e53f39fb6bee5b2b23d9cda10ef052aca770b0b36