知识卡片

普通读书笔记卡 · 8702.c · 生成模型:GPT-6

二值嵌入以低精度换空间

理解等级:中级

将每维量化为1bit并打包,512维由2048字节降至64字节,纯数值体积缩32倍。符号阈值、零值和校准影响结果;位运算比较的是压缩后的距离,不自动保持原浮点排序。

这张卡片看不懂?

已有反馈 · 0 条

    还没有人提交反馈。

    在整书笔记中的位置

    原书出处 · 1 处

    • · 查看原书依据

      Binary and Integer Embeddings An alternative to truncation is quantization. With binary and integer quantization, the number of vector dimensions remains the same, but each dimension is represented by fewer bits. Recall that typically embedding vectors are represented in float32, thus taking four bytes of memory per dimension. At the extreme level, the four bytes can be represented with just one bit, resulting in a

    参考来源

    - 位置:《Designing Large Language Model Applications》第11章《表示学习与嵌入》(来源项目:a19c7933-4eda-4176-aebd-30fe7e1ad292;源文件:extracted/Designing Large Language Model Applications/_epub-src/OEBPS/ch11.html) - 结论依据:摘录中关于二值嵌入以低精度换空间的定义、比较或条件支撑卡片;原书表述不严谨的范围已在覆盖记录逐项限定,不把案例结果外推为普遍结论。 - 原始内容:Binary and Integer Embeddings An alternative to truncation is quantization. With binary and integer quantization, the number of vector dimensions remains the same, but each dimension is represented by fewer bits. Recall that typically embedding vectors are represented in float32, thus taking four bytes of memory per dimension. At the extreme level, the four bytes can be represented with just one bit, resulting in a