知识卡片

普通读书笔记卡 · 8984 · 生成模型:GPT-6

截断对比散度带来采样偏差

理解等级:高级

若能精确从当前模型抽样,能量梯度的负相期望可作蒙特卡洛估计。经典对比散度常从数据出发运行有限步马尔可夫链,以得到较便宜的近似负样本;链未混合时通常有偏。它是梯度近似路线,不等同任何情况下精确最大似然梯度。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • · 查看原书依据

      Under our formalization of a learning problem as an objective, hypothesis space, and optimizer, contrastive divergence is an optimizer; it's an optimizer specifically built for max likelihood objectives over energy functions. Contrastive divergence tells us how to approximate the gradient of the likelihood function, which can then be plugged into any gradient descent method.

    参考来源

    - 位置:《Foundations of Computer Vision (Antonio Torralba, Phillip Isola etc.)》第32章《生成模型》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Foundations of Computer Vision (Antonio Torralba, Phillip Isola etc.)/_epub-src/OPS/xhtml/c032.xhtml) - 结论依据:摘录中关于截断对比散度带来采样偏差的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。 - 原始内容:Under our formalization of a learning problem as an objective, hypothesis space, and optimizer, contrastive divergence is an optimizer; it's an optimizer specifically built for max likelihood objectives over energy functions. Contrastive divergence tells us how to approximate the gradient of the likelihood function, which can then be plugged into any gradient descent method.