← 返回刚才阅读的位置

知识卡片

普通读书笔记卡 · 6717.b · 生成模型:GPT-6

词元嵌入进入卷积前需换轴

理解等级:中级

Embedding通常给出(B,L,E),Conv1d沿最后一维滑动且把中间维当通道,因此文本卷积一般换成(B,E,L)。只让数值维数碰巧能运算,可能变成沿嵌入坐标卷积;应按想处理的时间轴检查,而非任意reshape。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • · 查看原书依据

      Figure 7.16 shows what the nn.Embedding layer in PyTorch is doing behind the scenes. To orient you on how the 1D convolution slides over your data, the diagram shows three steps of a 2-length kernel stepping through your data. But how can a 1D convolution work on a sequence of 300D GloVe word embeddings? You just have to create a convolution kernel (filter) for each

    参考来源

    - 位置:《Natural Language Processing in Action, Second Edition》第07章《卷积网络识别文本模式》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Natural Language Processing in Action, Second Edition/_epub-src/OEBPS/Text/chapter-7.html) - 结论依据:摘录中关于词元嵌入进入卷积前需换轴的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。 - 原始内容:Figure 7.16 shows what the nn.Embedding layer in PyTorch is doing behind the scenes. To orient you on how the 1D convolution slides over your data, the diagram shows three steps of a 2-length kernel stepping through your data. But how can a 1D convolution work on a sequence of 300D GloVe word embeddings? You just have to create a convolution kernel (filter) for each