知识卡片
模型偏好排名不是通用智能百分比
成对偏好可汇总成相对评分,受到参与者、问题分布、展示和样本量影响。Elo分差或比例不是聪明百分比,某榜单也不是所有应用的客观能力;应补本领域人工测试,区分人类偏好与自动裁判。
这张卡片看不懂?
这张卡片已通过人工审查,不再接受失败反馈。
已有反馈 · 0 条
还没有人提交反馈。
在整书笔记中的位置
原书出处 · 1 处
- · 查看原书依据
For the reinforcement learning from human feedback (RLHF), these researchers and students at the Large Model Systems Organization (LMSYS) created an arena where the latest AI contenders could compete. Anyone can sign up and use the GUI to judge between pairs of contenders and help give chatbots intelligence ratings. When you dream up a challenging question and judge the chatbot answer, your rating is used
参考来源
- 位置:《Natural Language Processing in Action, Second Edition》第10章《真实世界中的大语言模型》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Natural Language Processing in Action, Second Edition/_epub-src/OEBPS/Text/chapter-10.html)
- 结论依据:摘录中关于模型偏好排名不是通用智能百分比的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。
- 原始内容:For the reinforcement learning from human feedback (RLHF), these researchers and students at the Large Model Systems Organization (LMSYS) created an arena where the latest AI contenders could compete. Anyone can sign up and use the GUI to judge between pairs of contenders and help give chatbots intelligence ratings. When you dream up a challenging question and judge the chatbot answer, your rating is used