原书出处
Designing Large Language Model Applications
Token acceptance rate This is the percentage of tokens generated by the draft model that are accepted. Typically, this does not reach 1, because if it did, there is no need to use the main larger model. Decoding speedup This refers to the reduction in latency between a model purely using autoregressive decoding versus one using speculative decoding. Constructing Draft Models How do we ensure
原书定位详情
来源:a19c7933-4eda-4176-aebd-30fe7e1ad292
正文位置:extracted/Designing Large Language Model Applications/_epub-src/OEBPS/ch09.html
EPUB SHA-256:1b4d2347fec2f949fcddfbaf4fad597a1ae20887a096cfae64c59a619b4781bf