知识卡片
伙伴数据隔离约束催生数千个小模型
YarnIt 除自营编织用品店外还有第三方伙伴市场。难点在于每个伙伴都要求把自己的数据与其他伙伴严格隔离,否则不允许上架商品;这意味着不能把所有商品与所有行为日志塞进一个主预测模型,而必须为每个伙伴训练单独的模型、把伙伴专属数据抽到隔离的存储库,共享数据仍可走公共特征库。由于伙伴数量规划在五千到五百万之间,架构必须针对「数千个极小模型」而非「少数几个大模型」优化:每天从主日志抽取、把前一日增量分离进各自目录或小特征库,训练前再合并,主模型与小模型都能快速训练,同时满足伙伴的访问隔离要求。作者点明:此前置阶段完成时,「舞台已搭好、故障条件已埋下」——同一份数据存在两个权威副本、周期性增量抽取,正是故事二故障的结构性根源。这也是理解 ML 事件必须看「前置条件」的最好例证。
这张卡片看不懂?
这张卡片已通过人工审查,不再接受失败反馈。
已有反馈 · 0 条
还没有人提交反馈。
在整书笔记中的位置
原书出处 · 1 处
- Incident Response · 查看原书依据
A notable constraint is that each of these partners requires that we separate their data from every other partner; otherwise, they won’t let us list their products.4 As a result, we’ll have to train a separate model per partner, and extract partner-specific data into isolated repositories, though we can still have a common feature store for shared data.
参考来源
- 位置:《Reliable Machine Learning》第11章《Incident Response》(源文件:extracted/Reliable Machine Learning/_epub-src/OEBPS/ch11.xhtml)
- 结论依据:原文明确阐述该主题,并给出定义、实践或反例。
- 原始内容:A notable constraint is that each of these partners requires that we separate their data from every other partner; otherwise, they won’t let us list their products.4 As a result, we’ll have to train a separate model per partner, and extract partner-specific data into isolated repositories, though we can still have a common feature store for shared data.