← 返回刚才阅读的位置

知识卡片

普通读书笔记卡 · 6667 · 生成模型:GPT-6

稀疏转稠密前先计算元素成本

理解等级:中级

41531×56404约为23.43亿个元素,以float64每元素8字节计算,仅数值区就约18.74GB,尚未计对象和额外副本。书中的2.4GB估计偏小;应尽量保留稀疏矩阵运算,不能因矩阵非零项少就随意转成稠密表。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • · 查看原书依据

      The TfidfVectorizer transformed your 41,531 documents into a sparse matrix with 56,404 columns, one column for each unique token that it found in your documents. You might be tempted to convert this sparse matrix into the more familiar NumPy array or pandas DataFrame, using the .todense() method, but be careful. When you convert a sparse matrix into a dense array, all those gaps representing unused

    参考来源

    - 位置:《Natural Language Processing in Action, Second Edition》第03章《词频与逆文档频率向量》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Natural Language Processing in Action, Second Edition/_epub-src/OEBPS/Text/chapter-3.html) - 结论依据:摘录中关于稀疏转稠密前先计算元素成本的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。 - 原始内容:The TfidfVectorizer transformed your 41,531 documents into a sparse matrix with 56,404 columns, one column for each unique token that it found in your documents. You might be tempted to convert this sparse matrix into the more familiar NumPy array or pandas DataFrame, using the .todense() method, but be careful. When you convert a sparse matrix into a dense array, all those gaps representing unused