知识卡片

普通读书笔记卡 · 6635.a · 生成模型:GPT-6

词元选专家需要处理负载失衡

理解等级:中级

token-choice为每个token挑少数专家,固定该token的分支数。若许多token集中到相同专家,可能出现排队、容量溢出或设备利用不均;应结合负载约束和路由策略评估,不能只看平均活跃专家数。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • · 查看原书依据

      tokens choose strategy, each token chooses k experts. k is typically a small number (~2). The disadvantage of using this strategy is the need for load balancing. If in a given input batch, most of the tokens end up using the same experts, then additional time is needed to finish the computation as we cannot benefit from the parallelization afforded by multiple experts.

    参考来源

    - 位置:《Designing Large Language Model Applications》第04章《架构与学习目标》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Designing Large Language Model Applications/_epub-src/OEBPS/ch04.html) - 结论依据:摘录中关于词元选专家需要处理负载失衡的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。 - 原始内容:tokens choose strategy, each token chooses k experts. k is typically a small number (~2). The disadvantage of using this strategy is the need for load balancing. If in a given input batch, most of the tokens end up using the same experts, then additional time is needed to finish the computation as we cannot benefit from the parallelization afforded by multiple experts.