原书出处
Designing Large Language Model Applications
K-L divergence of output probabilities In this method, our objective function is to minimize the K-L divergence between the output probability distribution of the teacher model and the student model. Internal representation similarity Conversely, instead of minimizing divergence, you can maximize similarity between aspects of the teacher and student model. This can be leveraged to perform layerwise distillation, where the internal representations of the teacher
原书定位详情
来源:a19c7933-4eda-4176-aebd-30fe7e1ad292
正文位置:extracted/Designing Large Language Model Applications/_epub-src/OEBPS/ch09.html
EPUB SHA-256:1b4d2347fec2f949fcddfbaf4fad597a1ae20887a096cfae64c59a619b4781bf