H2O-Lightning-4B is an open decision model built on Qwen3.5-4B that runs on unmodified vLLM. Version 1.2.2 takes up to four images per request.
By H2O.ai · Released 7 Oct 2026
A 4B open model built on Qwen3.5-4B that runs on unmodified vLLM. Version 1.2.2 takes up to four images per request.
H2O-Lightning-4B: A 4B open model built on Qwen3.5-4B that runs on unmodified vLLM.
Source: Announcement → Source: Hugging Face →
A lab notebook for speeding up one decision model, H2O Lightning 4B, served through llama.cpp on an RTX 3060 Ti. Each request takes 2.13 to 2.45 times its roofline floor. The first measured change, a llama.cpp pull request, cut latency by 1.94% with bit-for-bit identical answers; the bigger gains in its list of 15 targets are labelled predictions.
Moves llama.cpp decision serving code into a C library. Its recorded d1 request takes 678 ms on eight CPU threads. Saved CLI/server parity tests stay within 0.01 for Laya, exceed that probability tolerance for H2O and Clef, and reject the tested Perplexity package. On a Pixel 8a in Termux, the maker reports a 464 ms yes/no decision with d1-2B Q4_K_M on four threads and 2.1 GB RSS. Without repacking, that question takes 1,210 ms at 1.1 GB RSS. iOS and Capacitor app bindings remain plans.
Source: Licence → Source: Saved parity output → Source: Phone measurement and build notes →