原书出处
Designing Large Language Model Applications
Leveraging DEMix Layers Transformers can be made more modular by composing the model as a mixture of experts, as shown in Chapter 4. One way to divide the experts is to assign each expert a single domain. This removes the possibility of catastrophic forgetting when learning new domains because each expert is trained separately without affecting other experts. To implement this, Gururangan et al. propose
原书定位详情
来源:a19c7933-4eda-4176-aebd-30fe7e1ad292
正文位置:extracted/Designing Large Language Model Applications/_epub-src/OEBPS/ch07.html
EPUB SHA-256:1b4d2347fec2f949fcddfbaf4fad597a1ae20887a096cfae64c59a619b4781bf