知识卡片
投毒与提示注入作用路径不同
投毒改变训练数据以影响学得行为,提示注入则在运行上下文中诱导越过指令或信息边界。二者都影响系统安全但控制点不同;原文案例仅作风险线索,不假定所有泄露都能证明训练集已被改变。
这张卡片看不懂?
这张卡片已通过人工审查,不再接受失败反馈。
已有反馈 · 0 条
还没有人提交反馈。
在整书笔记中的位置
原书出处 · 1 处
- · 查看原书依据
data poisoning, which involves injecting malicious or biased data into the training set to corrupt the model. Recall the famous (or infamous) Microsoft Tay chatbot, which was taken down within a day of release because it learned inflammatory and offensive content from users who deliberately attempted to train it to produce such responses. Or more recently with the advent of LLMs, we’ve seen prompt injection
参考来源
- 位置:《Effective Machine Learning Teams (David Tan, Ada Leung, David Colls)》第01章《机器学习交付的挑战与路径》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Effective Machine Learning Teams (David Tan, Ada Leung, David Colls)/_epub-src/OEBPS/ch01.html)
- 结论依据:摘录中关于投毒与提示注入作用路径不同的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。
- 原始内容:data poisoning, which involves injecting malicious or biased data into the training set to corrupt the model. Recall the famous (or infamous) Microsoft Tay chatbot, which was taken down within a day of release because it learned inflammatory and offensive content from users who deliberately attempted to train it to produce such responses. Or more recently with the advent of LLMs, we’ve seen prompt injection