知识卡片

普通读书笔记卡 · 7007.k · 生成模型:DeepSeek-V4.1-Flash

统计偏差与校准图

理解等级:中级

金丝雀指标善于提示“出了大问题”,却分不出好与更好,正常也不代表万事大吉。统计偏差(统计义而非伦理义)比较“按预测期望看到的总量”与“实际看到的总量”是否相等,理想值为 0.00,偏离几个百分点往往就是警报;它常与切片分析搭配定位问题。但零偏差可被轻易伪造:一个对每个样本都返回观测均值的模型偏差为零却毫无信息量,所以必须看细粒度切片。校准图按模型分数分桶、逐桶算偏差,画观测发生率对预测概率,能看出系统性高估低估及输出两端是否可靠。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • Evaluating Model Validity and Quality · 查看原书依据

      The drawback of bias as a metric is that it is trivial to create a model with perfectly zero bias, but that is a terrible model. As a thought experiment, this could be done by having a model that just returns the average observed value for every example—zero bias in aggregate, but totally uninformative.

    参考来源

    - 位置:《Reliable Machine Learning》第05章《Evaluating Model Validity and Quality》(源文件:extracted/Reliable Machine Learning/_epub-src/OEBPS/ch05.xhtml) - 结论依据:原文明确阐述该主题,并给出定义、实践或反例。 - 原始内容:The drawback of bias as a metric is that it is trivial to create a model with perfectly zero bias, but that is a terrible model. As a thought experiment, this could be done by having a model that just returns the average observed value for every example—zero bias in aggregate, but totally uninformative.