← 返回刚才阅读的位置

知识卡片

普通读书笔记卡 · 7010.g · 生成模型:DeepSeek-V4.1-Flash

存储加载与无中断版本更新

理解等级:高级

服务模型有确定大小:离线模型可存磁盘、批作业里由特定二进制按需加载,成本是磁盘空间、加载的 I/O 与载入内存的 RAM(RAM 通常更贵也更受限);在线模型要放在专用机器的 RAM,高吞吐且延迟关键的还得在多台副本并行存多份。因多数模型终将按新数据重训(更新可到每周、每日、每小时甚至更频繁),必须在不停产的前提下换版本,两条主策略:一是给服务作业分配两倍 RAM,让新版本在旧版本仍在服务时先载入、就绪后热切换,有效但绝大多数非切换时间浪费 RAM;二是按百分比超配副本机,轮流下线一小部分(如 10%)逐批更新,更渐进也更便于错误检查与金丝雀。若还要支持 A/B 测试(多数团队都要),架构须能同时服务 A 与 B、甚至多种 B;支持几个版本、各占多少容量,是要在资源、系统复杂度与组织需求间权衡的架构决定。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • Serving · 查看原书依据

      The first is to allocate twice as much RAM for the serving jobs, so that the new version of the model can be loaded into the machine while the old one is still serving, and then hot-swapping which one is used once the new version is fully ready. This works well but is wasteful of RAM for the majority of the time when a model is not being loaded or swapped.

    参考来源

    - 位置:《Reliable Machine Learning》第08章《Serving》(源文件:extracted/Reliable Machine Learning/_epub-src/OEBPS/ch08.xhtml) - 结论依据:原文明确阐述该主题,并给出定义、实践或反例。 - 原始内容:The first is to allocate twice as much RAM for the serving jobs, so that the new version of the model can be loaded into the machine while the old one is still serving, and then hot-swapping which one is used once the new version is fully ready. This works well but is wasteful of RAM for the majority of the time when a model is not being loaded or swapped.