原书出处
Designing Large Language Model Applications
Matryoshka Representation Learning (MRL). In MRL, we first choose a set of truncation dimensions. For example a 1,024-dimension vector can have truncation dimensions 128, 256, 512, and 768. During the training process, we calculate the loss over each of the truncation dimensions as well as the full dimension. The losses are then added and weighted. In our example, the first 128 dimensions learn from the
原书定位详情
来源:a19c7933-4eda-4176-aebd-30fe7e1ad292
正文位置:extracted/Designing Large Language Model Applications/_epub-src/OEBPS/ch11.html
EPUB SHA-256:1b4d2347fec2f949fcddfbaf4fad597a1ae20887a096cfae64c59a619b4781bf