知识卡片
托管与自建按端到端成本比较
托管服务可提供优化内核与运维,自建则增加模型和数据控制。比较应包含网络、排队、负载、缓存、并发及维护成本,不能仅拿本机一次生成速度与服务宣传数字决定。
这张卡片看不懂?
这张卡片已通过人工审查,不再接受失败反馈。
已有反馈 · 0 条
还没有人提交反馈。
在整书笔记中的位置
- · 按任务适配模型
原书出处 · 1 处
- · 查看原书依据
LLM Inference APIs While you can deploy an LLM yourself, modern-day inference consists of so many optimizations, many of them proprietary, that it takes a lot of effort to bring your inference speeds up to par with commercially available solutions. Several inference services like Together AI exist that facilitate inference of open source or custom models either through serverless endpoints or dedicated instances. Another option
参考来源
- 位置:《Designing Large Language Model Applications》第05章《按任务适配模型》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Designing Large Language Model Applications/_epub-src/OEBPS/ch05.html)
- 结论依据:摘录中关于托管与自建按端到端成本比较的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。
- 原始内容:LLM Inference APIs While you can deploy an LLM yourself, modern-day inference consists of so many optimizations, many of them proprietary, that it takes a lot of effort to bring your inference speeds up to par with commercially available solutions. Several inference services like Together AI exist that facilitate inference of open source or custom models either through serverless endpoints or dedicated instances. Another option