知识卡片
模型缓存与渐进发布
模型文件大,每次实例启动都从远端下载会扩大冷启动;共享缓存可减少重复传输,但应按模型版本和完整性组织。模型变化会不可预测地影响输出质量,先小范围暴露、观察再扩展,比一口气替换全部用户更可控。格式互通也应验证算子和运行时支持,不能把ONNX理解为任意模型均可无损跨框架。
这张卡片看不懂?
这张卡片的反馈已满 10 条,暂时无法继续提交。
已有反馈 · 0 / 10 条
还没有人提交反馈。
在整书笔记中的位置
原书出处 · 1 处
- Chapter 15. AI Inference and Serving · 查看原书依据
cache the model locally, either within a shared filesystem or an in-memory key-value store. Once you cache your model, all of your instances for inference can load from the same cached model. When you restart your application because of failures or new software releases, you don’t have to
参考来源
- 位置:《Designing Distributed Systems, 2nd Edition》第15章《Chapter 15. AI Inference and Serving》(来源项目 book-sources;源文件:extracted/Designing Distributed Systems, 2nd Edition/_epub-src/OEBPS/ch15.html)
- 结论依据:原文讲模型分发、缓存和逐步扩大新模型曝光;保留兼容验证边界。
- 原始内容:cache the model locally, either within a shared filesystem or an in-memory key-value store. Once you cache your model, all of your instances for inference can load from the same cached model. When you restart your application because of failures or new software releases, you don’t have to