docs(readme): mechanical fixes — expert count, IO_THREADS, line counts, desktop/

- Expert count 21,504 -> 19,456 (75x256 + MTP head), matching the rest of the repo.
- Replace phantom IO_THREADS with PIPE_WORKERS (default 8); state the pool only
  engages under PIPE=1.
- Drop rotting precise line-count claims (glm.c ~2,400; web ~390) — keep the prose.
- Add the desktop/ directory to the repo-layout section.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
KingIcyCreamProjects
2026-07-15 01:21:15 -05:00
parent 3fd47b7bbd
commit 4059e10761
+8 -7
View File
@@ -48,9 +48,9 @@ brightness is routing heat, and every expert routed in a turn flashes white. Hov
A 744B Mixture-of-Experts model activates only ~40B parameters per token — and only ~11 GB of those change from token to token (the routed experts). So:
- the **dense part** (attention, shared experts, embeddings — ~17B params) stays **resident in RAM at int4** (~9.9 GB);
- the **21,504 routed experts** (75 MoE layers × 256 experts + the MTP head, ~19 MB each at int4) live **on disk** (~370 GB) and are **streamed on demand**, with a per-layer LRU cache, an optional pinned hot-store, and the OS page cache as a free L2.
- the **19,456 routed experts** (75 MoE layers × 256 experts + the MTP head, ~19 MB each at int4) live **on disk** (~370 GB) and are **streamed on demand**, with a per-layer LRU cache, an optional pinned hot-store, and the OS page cache as a free L2.
The engine is a single C file (`c/glm.c`, ~2,400 lines) plus small headers. No BLAS, no Python at runtime, no GPU required (an opt-in CUDA tier for pinned experts exists — see below).
The engine is a single C file (`c/glm.c`) plus small headers. No BLAS, no Python at runtime, no GPU required (an opt-in CUDA tier for pinned experts exists — see below).
## What's implemented
@@ -455,8 +455,8 @@ CUDA, CPU hot-store, and CUDA hot-expert execution with identical replay tokens.
### Web interface
`web/` contains a community-contributed browser UI (React + TypeScript, ~390
lines of source, a pure API client — it never touches the engine directly):
`web/` contains a community-contributed browser UI (React + TypeScript, a pure
API client — it never touches the engine directly):
```bash
cd web
@@ -493,10 +493,10 @@ Disk is an immutable recovery source, not a normal decode target. If the plan
leaves cold expert bytes on disk, speed depends on cache hit rate; output
quality does not.
Cold expert reads use a deferred pipeline: resident RAM/VRAM experts execute
Cold expert reads can use a deferred pipeline: resident RAM/VRAM experts execute
while missing experts are loaded in a bounded background I/O pool, then the
cold results join before the layer completes. `IO_THREADS=n` overrides the
default eight loader threads when foreground work exists. Profiling reports
cold results join before the layer completes. The pool engages only under
`PIPE=1`; `PIPE_WORKERS=n` sets its worker count (default 8). Profiling reports
both disk service time and the smaller foreground-visible wait time so overlap
is explicit rather than credited as unexplained speedup.
@@ -657,6 +657,7 @@ c/
├── scripts/ long-running conversion helpers
└── tests/ dependency-free C and Python tests
web/ browser UI (pure OpenAI-API client, community-maintained)
desktop/ Tauri v2 desktop shell wrapping the web UI
```
The runtime path intentionally stays flat and readable: `glm.c` plus its small