知识卡片
融合分支须对齐输出形状
跨注意力输出与文本自注意力输出可相加或加权融合,但文本长度与最终通道维度需相容,必要时先投影。可学习权重只是一个设计,不是所有视觉语言模型必须采用的固定架构;加权相加也不保证两模态都被有效利用。
这张卡片看不懂?
这张卡片已通过人工审查,不再接受失败反馈。
已有反馈 · 0 条
还没有人提交反馈。
在整书笔记中的位置
- · 视觉与语言
原书出处 · 1 处
- · 查看原书依据
where Tout,self is regular self-attention on the input text token Tt, and wcross and wself are optional learnable weights. The text decoder is an autoregressive model (section 32.7) that uses causal attention (section 26.10) to predict the next word in the sentence given the previous words generated so far. In this strategy, the input to the text decoder is the tokens representing the previous words
参考来源
- 位置:《Foundations of Computer Vision (Antonio Torralba, Phillip Isola etc.)》第51章《视觉与语言》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Foundations of Computer Vision (Antonio Torralba, Phillip Isola etc.)/_epub-src/OPS/xhtml/c051.xhtml)
- 结论依据:摘录中关于融合分支须对齐输出形状的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。
- 原始内容:where Tout,self is regular self-attention on the input text token Tt, and wcross and wself are optional learnable weights. The text decoder is an autoregressive model (section 32.7) that uses causal attention (section 26.10) to predict the next word in the sentence given the previous words generated so far. In this strategy, the input to the text decoder is the tokens representing the previous words