Use these snippets for careful, citation-oriented outreach. The goal is to attract readers who may reuse, benchmark, or cite the method.
For a more complete copy-ready package, including list-maintainer blurbs,
benchmark calls to action, and sharing boundaries, use
share_kit.md.
me_fasttext is a FastText-derived C++ prototype that replaces hash-bucketed
subword ids with exact trie-backed ids, then exports a compact mmap serving
index for large-vocabulary NLP.
I released me_fasttext, a memory-efficient FastText-derived C++ prototype for
large-vocabulary NLP.
The core idea: train with exact trie-backed word/subword ids instead of opaque
hash buckets, then export a compact mmap serving index via conservative n-gram
row sharing and mark-compact layout.
Useful when you still want FastText-style OOV behavior for retrieval, ranking,
or long-tail text features, but need a smaller serving footprint.
Code: https://github.com/initial-d/me_fasttext
Paper: https://arxiv.org/abs/2506.01254
FastText's hash buckets are convenient, but they make character n-gram identity
opaque: unrelated subwords can collide into the same row.
me_fasttext explores an exact-first, compact-later design:
1. assign words and UTF-8 character n-grams exact trie-backed ids
2. train FastText-style word/subword embeddings
3. merge only structurally related n-gram rows with very close vectors
4. rewrite live rows into a compact mmap-ready serving artifact
The paper reports a large Chinese vocabulary benchmark with 30.1M words and
287.4M n-grams, reducing model memory from 145.2 GB to 28.9 GB while preserving
downstream quality within 0.3 points of the hash-free model.
Code: https://github.com/initial-d/me_fasttext
Paper: https://arxiv.org/abs/2506.01254
Engineering note: https://github.com/initial-d/me_fasttext/blob/main/docs/engineering_note.md
I finally wrote down the design idea behind me_fasttext:
FastText-style subword models are not only NLP models. At large vocabulary
scale, they become memory-layout problems.
me_fasttext treats character n-gram embeddings as structured memory objects:
train first with exact trie-backed subword identities, then rewrite retained
rows into a compact mmap serving artifact using a mark-compact-style export
step.
The interesting part is the bridge between three older ideas:
1. classical dynamic memory management
2. trie-backed lexical identity
3. FastText's character n-gram statistics
Design essay:
https://github.com/initial-d/me_fasttext/blob/main/docs/memory_management_for_ai_embeddings.md
Code:
https://github.com/initial-d/me_fasttext
I have been reframing me_fasttext as an early compact-inference prototype for
lexical embeddings.
Before the current LLM-serving wave, this project already explored a familiar
deployment problem: keep useful embedding behavior while reducing serving
memory, cold-load time, and artifact size.
The design:
1. train FastText-style embeddings with exact trie-backed word/subword ids
2. keep character n-gram identity inspectable before compression
3. merge only selected structurally related rows after training
4. rewrite live rows into a compact mmap-ready serving artifact
It is not trying to replace dense embedding models. It is a small lexical
inference layer for OOV-heavy retrieval, entity search, long-tail Chinese text,
and low-memory first-stage ranking.
Code: https://github.com/initial-d/me_fasttext
Paper: https://arxiv.org/abs/2506.01254
Inference note:
https://github.com/initial-d/me_fasttext/blob/main/docs/inference_optimization_note.md
For the current runnable storage experiment and source audit, prefer Engram-focused launch copy. The older snippets below describe broad context; they are not evidence of lossless export or Engram integration.
Use this when discussing Engram-style conditional memory or indexed memory systems. Keep the framing cautious: adjacent design pressure, not lineage.
DeepSeek's Engram is a useful reminder that language memory does not always
need to live in the same dense computation path.
That is also the narrow systems lens behind me_fasttext, at a much smaller and
more inspectable scale:
- keep word and character n-gram identities explicit
- use trie-backed lookup instead of opaque hash buckets
- compact learned rows after training
- serve the result as a mmap-friendly lexical memory artifact
This is not an LLM memory module, and it is not a claim of lineage. It is a
compact lexical example of the same broader design pressure: indexed memory,
sparse lookup, and careful reporting of memory, latency, coverage, and quality.
Context note:
https://github.com/initial-d/me_fasttext/blob/main/docs/conditional_memory_context.md
Code:
https://github.com/initial-d/me_fasttext
DeepSeek 把 Engram / conditional memory 带回视野以后，我觉得可以重新看
me_fasttext 这个小项目。
它不是 LLM memory module，也不是说两者有直接继承关系。更准确的说法是：
它们面对的是相邻的系统压力。
有些语言记忆不一定都要走昂贵的 dense computation path。比如长尾词、实体名、
n-gram-like signal，可以先保留显式 identity，再用索引结构查找，最后为 serving
做 compact layout。
me_fasttext 做的是 FastText / lexical embedding 这一层的小型版本：
- trie-backed exact word/subword ids
- training 后再做保守 row sharing
- mark-compact-style row rewriting
- mmap-friendly `.z` serving artifact
我写了一个克制的 context note，只讲 adjacent design pressure，不碰瓷 lineage：
https://github.com/initial-d/me_fasttext/blob/main/docs/conditional_memory_context.md
我整理了一篇 `me_fasttext` 的中文技术回顾：
《我把动态内存管理用在了 FastText 的 n-gram embedding 上》
这个项目表面上是一个 memory-efficient FastText 变体，但真正的设计点是：
把 FastText 的大规模 n-gram embedding 矩阵看成一个结构化内存对象。
先用 trie 保留精确 subword identity，再在训练后用类似 mark-compact 的方式
把向量行整理成紧凑、可 mmap serving 的模型 artifact。
我觉得它最有意思的地方，是把动态内存管理、字典树、FastText 的 n-gram
语言统计现象接在了一起。
文章：
https://github.com/initial-d/me_fasttext/blob/main/docs/technical_retrospective_zh.md
代码：
https://github.com/initial-d/me_fasttext
- Memory-Efficient FastText with Exact Trie N-Gram IDs and mmap Serving
- Replacing FastText Hash Buckets with Trie-Backed Subword IDs
- Mark-Compact Layouts for FastText-Style Embedding Serving
- Memory Management for FastText-Style Subword Embeddings
- I built a memory-efficient FastText variant with trie-backed subword IDs
- Exact subword IDs + mmap serving for FastText-style embeddings
- Revisiting FastText for OOV-heavy retrieval and low-memory serving
- Applying mark-compact memory management ideas to subword embeddings
Hi, thanks for maintaining this list.
Would you consider adding `me_fasttext`?
It is a FastText-derived C++ research prototype for memory-sensitive
large-vocabulary NLP. Instead of hashing character n-grams into fixed buckets,
it assigns exact trie-backed ids, then exports a compact mmap serving index via
conservative row sharing and mark-compact layout.
Repo: https://github.com/initial-d/me_fasttext
Paper: https://arxiv.org/abs/2506.01254
Disclosure: I maintain the project. I think it fits best under word embeddings,
retrieval infrastructure, or model compression depending on your taxonomy.
Happy to adjust or withdraw if it is out of scope.Hi,
I thought this might be relevant to your work on embeddings / retrieval /
efficient NLP systems.
I released me_fasttext, a FastText-derived C++ prototype that replaces
hash-bucketed character n-gram ids with exact trie-backed ids, then exports a
compact mmap serving index. The goal is to keep FastText-style OOV behavior
while reducing large-vocabulary serving cost.
Paper: https://arxiv.org/abs/2506.01254
Code: https://github.com/initial-d/me_fasttext
If you try it on another corpus, I would be especially interested in benchmark
reports that include memory, load time, query latency, and downstream quality.
Use this only where appropriate:
If the exact-id-before-compression design, benchmark protocol, or mmap serving
layout is useful in your own work, please cite the arXiv paper:
https://arxiv.org/abs/2506.01254
- Do not claim it replaces transformer embeddings.
- Do not call it a complete RAG framework.
- Do not describe it as a garbage collector implementation.
- Do not imply the reported compression ratio is universal.
- Do not post the same text across many communities on the same day.