知识卡片
评估等于指标加分布
作者给出关键定义:一次评估由两个部件组成——性能指标(准确率、精确率、召回率、ROC 曲线下面积等)和数据分布(如留出集、“雪天道路图片”或“挪威用户的毛线店查询”)。指标和分布必须成对出现,评估只说明模型在该分布上按该指标算出的表现。行业里常说“这个模型准确率更高”却不交代分布,这种省略很危险:2010 年代末许多公平性问题都可追溯到测试时没有认真考虑数据分布的具体构成。听到“准确率更好”,最该追问“在哪个分布上”。
这张卡片看不懂?
这张卡片已通过人工审查,不再接受失败反馈。
已有反馈 · 0 条
还没有人提交反馈。
在整书笔记中的位置
原书出处 · 1 处
- Evaluating Model Validity and Quality · 查看原书依据
Now, what is an evaluation? An evaluation consists of two key components: a performance metric and a data distribution. Performance metrics are things like accuracy, precision, recall, area under the ROC curve, and so on—we will talk about a few of these later on if they are not already familiar.
参考来源
- 位置:《Reliable Machine Learning》第05章《Evaluating Model Validity and Quality》(源文件:extracted/Reliable Machine Learning/_epub-src/OEBPS/ch05.xhtml)
- 结论依据:原文明确阐述该主题,并给出定义、实践或反例。
- 原始内容:Now, what is an evaluation? An evaluation consists of two key components: a performance metric and a data distribution. Performance metrics are things like accuracy, precision, recall, area under the ROC curve, and so on—we will talk about a few of these later on if they are not already familiar.