⁂ asterism + inferenZ
A local agent and a CPU inference engine.
asterism is a personal agent that runs on your computer, used from Telegram or the terminal. inferenZ runs a 125B-parameter mixture-of-experts model on a laptop CPU, reading the experts from the SSD. asterism can use any OpenAI-compatible server as a model backend, inferenZ included.
CLocal inferenceMIT
01 / AGENT
⁂ asterism
Personal agent, local first.
- Interface
- Telegram (text, voice notes, photos, files) and a terminal chat.
- Models
- Local GGUF models through llama.cpp, in three sizes: base (8 GB RAM, no GPU), standard (8 GB VRAM), plus (16 GB VRAM). Optionally ChatGPT for planning and answers; coding tasks delegated to Codex or Claude Code.
- Tools
- Files, search, code, git, build and test, processes, delegation. Each call is a separate process with declared permissions, a time limit and an output cap; strict sandbox on Linux.
- Approval
- Writes, commands and network access need confirmation. After untrusted input (web pages, forwarded messages, other agents) every action does.
- Secrets
- Encrypted vault. The model sees
{{secret:NAME}}; only the tool the secret is stored for gets the value. - Memory
- Verbatim log of every exchange. Notes about the user, general facts and each project are extracted while idle and retrieved within a fixed budget; unused notes decay.
- Projects
- A folder with its own Telegram thread. Scheduled routines:
/routine add every day at 8 | summarize my mail. - Network
- No listening ports; Telegram is polled.
- Build
- One executable, CMake. Python and JavaScript SDKs. Tests run without models or network.
$ curl -fsSL https://raw.githubusercontent.com/gslf/asterism/main/install.sh | sh > irm https://raw.githubusercontent.com/gslf/asterism/main/install.ps1 | iex # Windows
02 / ENGINE
inferenZ
Single-model inference engine, CPU only.
- Model
- Qwen3.8-Flash-Next: 125B parameters, 6B active per token. 48 layers (36 Gated DeltaNet, 12 sparse QSA attention), 512 experts per layer with top-10 routing, MTP layer.
- Requires
- Linux, x86-64 with AVX2 and AVX-VNNI, about 64 GB of RAM. No GPU.
- Weights
- 124 GB (Q4 experts, Q8 dense). Dense weights stay in RAM, experts are read from the SSD.
- Streaming
- One 4 KiB-aligned blob per expert, read with io_uring and O_DIRECT into an LRFU cache sized to the available memory. The next layer's router runs one layer ahead to prefetch. Cache scores persist across restarts.
- Decoding
- MTP speculative decoding with exact speculative sampling. Experts are combined in a fixed order, so the output does not depend on I/O timing.
- Speed
- About 8 tokens/s on a laptop with 62 GB of RAM and a 24 GiB cache.
- Interfaces
- OpenAI-compatible API (chat, streaming, reasoning, tool calls), web UI with live diagnostics, terminal chat.
- Validation
- Compared layer by layer with the reference implementation. Tokenizer and chat template are byte-exact.
- Build
- C, model shapes fixed at compile time, kernels written for this model only.
$ make $ ./inz serve # http://127.0.0.1:8080, converted model required