Local decision models and their runtimes
Local decision models run through different runtimes. Start with your machine and the model’s loading code: MLX, ONNX and GGUF packages need the runtime that understands their decision outputs.
A first local request with JevAlt
JevAlt provides one documented CPU route from download to a typed decision. Use Python 3.11 or newer. Its default English Deem-4B Q4_K_M file is 2.71 GB; the maker measured 3.04 GB peak RAM at 4k context on one configuration. Allow memory for the rest of your application. Code and weights are Apache-2.0; dataset terms remain separate.
JevAlt model and limits · Checked server project
Source: github.com · Source: huggingface.co
Install the server and model loader
Install the serve and gguf extras. This command follows the maker's recipe; its dependency versions are not fully pinned. The gguf extra uses llama-cpp-python, which needs to install successfully on your platform.
pip install "jevalt[serve,gguf] @ git+https://github.com/mertkayacs/jevalt"Start the English model
Run the server and leave this terminal open. The first start downloads Deem-4B Q4_K_M from Hugging Face and listens on 127.0.0.1:8000. The server performs inference locally after download.
jevalt serveAsk a choice question
Open another terminal and send this support ticket. Read the selected option and per-option probabilities under answers.team. This is the maker's example request; no particular probability is promised. A server or loading error is not a decision and should stop the workflow.
curl -s http://127.0.0.1:8000/v1/systemone -H 'Content-Type: application/json' -d '{
"state": "I was charged twice for March. Please refund the duplicate.",
"questions": {"team": {"type": "choice",
"instructions": "Which team should handle this ticket?",
"criteria": {"billing": "payments, refunds", "technical": "bugs, outages",
"sales": "prices, contracts"}}}}'Check one answer before automating
The maker also publishes live requests, including mistakes and missing facts. Those saved records demonstrate an author-run model; the setup here has not been independently rerun. A request that only mentions late arrival cannot establish a hotel room type. Optional unknown answers need abstain enabled, and the maker's 11-case missing-fact test gives both JevAlt and its base 9 unknown choices. Start with labelled examples from your own task before choosing an action threshold.
Evaluate your own task · Other endpoint examples
Source: huggingface.co · Evidence: Mert Kaya · Deem-4B card and recorded requests · October 2026, catch-up discovery 8 Nov · Source ↗
Apple Silicon: Laya or a d1 integration
laya-mlx runs Laya natively through MLX, without PyTorch or a cloud API. Its Snake demo includes a safety layer that can override a model move. For an OpenClaw agent, openclaw-d1-local loads LiquidAI/d1-Omni-2B-MLX-8bit and supplies a native provider plus a bridge for OpenClaw’s TypeSafe integration. Despite its repository name, Kev-claw-decisionmaking loads d1-Omni. These are different integration routes, so use the instructions for the specific model and application.
Browser: Laya or SemIf
Laya in the browser uses ONNX Runtime Web, with WASM on CPU by default. It downloads the model before local inference. SemIf uses pinned GGUF builds through wllama and reads option logits. SemIf reproduces an interface pattern and is independent of TypeSafe; neither route downloads TypeSafe’s private Jev weights. Browser parity and task accuracy answer different questions: Laya’s reported 97.9% top-answer agreement with PyTorch on 48 questions per quantized build does not make those answers correct.
Laya browser project · SemIf model facts
Source: github.com · Source: github.com · Source: github.com
Local HTTP API: Ollama or Noullama
Ollama v0.35 introduced /v1/systemone for choice, yes/no and score questions, with Nimble and Tev1 at launch. Noullama is a desktop runtime that finds decision-model GGUF files, runs them through llama.cpp and serves a local typed-question API. Check each project’s supported models before downloading weights. A GGUF file alone does not establish compatibility with a particular decision head or API.
Ollama runtime facts · Noullama project · Nimble model facts · Tev1 model facts
An ordinary local LLM: decision-cli
decision-cli reads answer-token probabilities from one forward pass of an ordinary MLX model on Apple Silicon. It exposes a CLI, batch mode and a Jev-like API, without fine-tuning. Its published scores use 231 public JevBench questions; they do not establish performance on private workloads or every possible question type.
What the hardware records tell you
VENDOR-REPORTED: openclaw-d1-local uses about 3 GB of resident RAM and takes about 200 ms per warm batch on its author’s M4 Mac mini. laya-mlx reports 13.4 ms median for a short English typed decision and 7.4 ms for its multilingual checkpoint on an M3 Max. These are separate models and workloads, not a speed ranking. Laya’s browser project warns that quantized confidence values are approximate because calibration used the full-precision model. These isolated records cannot tell you whether a particular Q8 file fits a 6 GB GPU at your chosen context and batch size.
OpenClaw’s reported configuration · Laya browser parity checks
Source: github.com · Source: github.com · Source: github.com
Before giving it real work
Choose the route that fits your application, then follow its own repository’s setup and loading instructions. Keep the model revision and runtime version with a small labelled sample of your task. Measure correct answers, errors among accepted answers, latency and peak memory at the context and batch size you expect to use. Recheck after changing precision. This guide identifies documented routes; it does not supply a locally reproduced installation recipe or a memory-sizing calculator.
Evaluate on your own task · Decision models versus classifiers · Compare model access and inputs
One Android phone measurement
The kneejerk maker reports d1-2B Q4_K_M running inside Termux on a Google Pixel 8a with 8 GB RAM, using four CPU threads. One 44-token yes/no question takes 464 ms; a four-option, 87-token question takes 1,100 ms. Process RSS is 2.1 GB with weight repacking. Without repacking, the yes/no question takes 1,210 ms at 1.1 GB RSS. These are two small requests on one phone, without a sustained-throughput, battery or thermal test. The code runs locally after the model download; the iOS Swift binding and Capacitor integration are unfinished.
kneejerk project and parity limits · d1 family and licence · Recorded hardware and quantization limits
Run an image question on Raspberry Pi 4
The d1 assembly project gives one documented route for a physical Raspberry Pi 4B with 4 GB RAM. Use 64-bit Raspberry Pi OS, GNU Make, GCC, glibc development headers and Python 3.10 or newer. Allow about 3 GB of disk space for weights and generated assets. Download d1-2B-Q4_K_M.gguf and mmproj-d1-2B-Q8_0.gguf from Liquid AI’s official GGUF repository. The preparation script validates the expected checkpoint bytes. Run the commands below from the project root after placing both files in models/.
python3 -m venv .venv
. .venv/bin/activate
python3 -m pip install -r requirements.txt
bash tools/prepare-model.sh models/d1-2B-Q4_K_M.gguf models/mmproj-d1-2B-Q8_0.gguf
make
python3 research/d1-640x480/decide.py models/d1-2B-Q4_K_M.gguf models/mmproj-d1-2B-Q8_0.gguf photo.jpg examples/choice.jsonLocal hardware and quantization limits
Source: github.com · Source: github.com
Keep the image, prompt and timing limits
The helper resizes and pads one image to 640 by 480 pixels. The benchmarked build has a 512-position context, so keep the prompt short. Choice questions update 48 selected image positions after attention ranking; direct counts keep all 300 because the sparse path missed the count target. In four paired Pi choice probes the maker reports medians of 28.02 seconds for assembly and 70.19 seconds for llama.cpp. Warm filesystem caches, background CPU load and different image-decode boundaries limit that comparison. The 80-answer accuracy record uses an x86 llama.cpp reference, checked against the eight Pi timing probes. These are the maker’s runs; this guide does not report a new installation or inference.
Engine, licence and run records · d1 family and model terms · Recorded Pi timing evidence
Evidence: DarvinX · d1-2B ARM64 engine · 11 Nov 2026 · Source ↗