原书出处
Designing Large Language Model Applications
Takes an older checkpoint of the model and during training compares the KL-divergence between the older and the current representations and penalizes it. Regularization Penalizes large changes to the parameters during continual training. Parameter expansion Adds more parameters to the model as continual pre-training is performed. This can be done by increasing either the width or the depth of the model. For a more comprehensive
原书定位详情
来源:a19c7933-4eda-4176-aebd-30fe7e1ad292
正文位置:extracted/Designing Large Language Model Applications/_epub-src/OEBPS/ch07.html
EPUB SHA-256:1b4d2347fec2f949fcddfbaf4fad597a1ae20887a096cfae64c59a619b4781bf