知识卡片

普通读书笔记卡 · 7012.k · 生成模型:DeepSeek-V4.1-Flash

只收到被选中商品的反馈与探索缺失

理解等级:高级

依赖反馈回路学习新趋势的系统,若压根没有「先试一件新东西」的机制,可能完全错过新趋势——可联想哄孩子第一次尝试新食物。更具体些:帮用户推荐毛线商品的模型,会拿用户对被选中展示商品的反应(点击、页面浏览、购买)来训练,于是模型只收到关于「被选中展示给用户」的商品的反馈,收不到关于未被选中商品的反馈(技术上属上下文老虎机设定,存在探索与利用的权衡)。可以想象一款新颜色有机羊驼毛线,用户其实很想买,但模型没有既往数据,于是继续推荐此前非有机的产品并对自己的遗漏浑然不觉。发现不了新东西已经不好,更糟的行为也可能发生:股票预测模型若观察到外部实体误发一笔大额卖出,可能预测市场将跌而主张卖出大部分持仓,卖出足够大就真压低市场、使模型更激进地想卖,而完全不相关组织的其他模型也可能看到同一信号决定卖出,形成强化回路乃至整体市场闪崩(这一极端情形确实发生过,见 Flash Crash)。不必身处多模型竞争的大市场也会遇到:yarnit.ai 里若一个模型负责推荐商品、另一个负责决定何时给折扣或优惠券,而推荐模型把购买行为用作训练信号、折扣又影响购买行为,两模型之间就有反馈回路,改一个会影响另一个。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • Continuous ML · 查看原书依据

      The model will receive feedback about the products that were selected to be shown to the user, but will not get feedback about products that were not selected.4 It is easy to imagine that a new wool product, such as a new color of organic alpaca yarn, might be something that users would love to purchase, but for which the model has no previous data. In this case, the model might continue recommending previous nonorganic produc

    参考来源

    - 位置:《Reliable Machine Learning》第10章《Continuous ML》(源文件:extracted/Reliable Machine Learning/_epub-src/OEBPS/ch10.xhtml) - 结论依据:原文明确阐述该主题,并给出定义、实践或反例。 - 原始内容:The model will receive feedback about the products that were selected to be shown to the user, but will not get feedback about products that were not selected.4 It is easy to imagine that a new wool product, such as a new color of organic alpaca yarn, might be something that users would love to purchase, but for which the model has no previous data. In this case, the model might continue recommending previous nonorganic produc