原书出处
Designing Large Language Model Applications
Gradient checkpointing Gradient checkpointing helps save memory at the cost of more compute. During the forward pass of the backpropagation algorithm, activations are computed and saved in memory so that they can be used in the backward pass. What if we did not save all of the activations? The missing activations could be recalculated on the fly during the backward pass. This does cost us
原书定位详情
来源:a19c7933-4eda-4176-aebd-30fe7e1ad292
正文位置:extracted/Designing Large Language Model Applications/_epub-src/OEBPS/ch06.html
EPUB SHA-256:1b4d2347fec2f949fcddfbaf4fad597a1ae20887a096cfae64c59a619b4781bf