知识卡片

普通读书笔记卡 · 2931.e.2.b.5.c.1.c

AI负载不只看请求频率

理解等级:中级

相同请求次数可能包含不同长度输入和输出,消耗明显不同。噪声邻居策略需把请求规模、模型和频率纳入观察与约束,而不是只限定每秒调用次数;具体负载模型应由实际服务数据验证。

这张卡片看不懂?

已有反馈 · 0 / 10 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • GenAI and Multi-Tenancy · 查看原书依据

      neighbor footprint of your environment. With GenAI workloads, the complexity of requests and responses also have some correlation to the level of load that is being placed on the service. This requires us to add another layer to our noisy neighbor model, introducing constructs that can evaluate the number of tokens and complexity of individual requests, ensuring that tenants aren’t

    参考来源

    - 位置:《Building Multi-Tenant SaaS Architectures》第16章《GenAI and Multi-Tenancy》(来源项目 a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Building Multi-Tenant SaaS Architectures/_epub-src/OEBPS/ch16.html) - 结论依据:原文具体说明相同请求次数可能包含不同长度输入和输出,消耗明显不同;本卡据此归纳机制及适用边界。 - 原始内容:neighbor footprint of your environment. With GenAI workloads, the complexity of requests and responses also have some correlation to the level of load that is being placed on the service. This requires us to add another layer to our noisy neighbor model, introducing constructs that can evaluate the number of tokens and complexity of individual requests, ensuring that tenants aren’t