Needle
Needle 2 is an open 45M-parameter model for tool calling, device use and structured extraction. The whole model is a single 14MB binary that runs a full session in about 28MB of RAM. It is built on our Simple Attention Network findings, compressed to CQ2-bit with Cactus Quants, and baked into its own engine. On the benchmarks below, Needle 2 trades wins with other small models like FunctionGemma 270M, LFM2.5 230M and Apple FM, at 5x to 70x smaller, and 2 bits against their f16. This repository is the Python package: inference, LoRA fine-tuning, and export. pip install cactus-needle, describe your tools, and call them from Python. The inference engine is fetched once from Hugging Face and cached; there is nothing else to build. Self-contained: weights baked into a single 14MB engine; no separate model files to manage, and inference does no network. Simple contract: tool calls come back as structured data, text in, JSON out; a byte-level grammar compiled from your schemas constrains every token. Confidence-gated: every response carries a calibrated confidence score from a learned head; set a threshold, act above it, escalate below it. Tool retrieval: declare a large catalogue and a built-in retrieval head renders only the top five tools per turn, with the grammar constrained to that subset. Bounded memory: a 256-token sliding window with the tools pinned as KV sinks, so total memory stays near 28MB no matter how long the conversation runs. Weights: huggingface.co/Cactus-Compute/needle2 · source: github.com/cactus-compute/needle.
View Needle on GitHub
Needle 2 is an open 45M-parameter model for tool calling, device use and structured extraction. The whole model is a single 14MB binary that runs a full session in about 28MB of RAM. It is built on our Simple Attention Network findings, compressed to CQ2-bit with Cactus Quants, and baked into its own engine. On the benchmarks below, Needle 2 trades wins with other small models like FunctionGemma 270M, LFM2.5 230M and Apple FM, at 5x to 70x smaller, and 2 bits against their f16.
This repository is the Python package: inference, LoRA fine-tuning, and export. pip install cactus-needle, describe your tools, and call them from Python. The inference engine is fetched once from Hugging Face and cached; there is nothing else to build. Self-contained: weights baked into a single 14MB engine; no separate model files to manage, and inference does no network. Simple contract: tool calls come back as structured data, text in, JSON out; a byte-level grammar compiled from your schemas constrains every token. Confidence-gated: every response carries a calibrated confidence score from a learned head; set a threshold, act above it, escalate below it. Tool retrieval: declare a large catalogue and a built-in retrieval head renders only the top five tools per turn, with the grammar constrained to that subset. Bounded memory: a 256-token sliding window with the tools pinned as KV sinks, so total memory stays near 28MB no matter how long the conversation runs.
Weights: huggingface.co/Cactus-Compute/needle2 · source: github.com/cactus-compute/needle.
Needle at a glance
| Stars | 13k |
|---|---|
| Forks | 868 |
| Language | Python |
| License | Apache-2.0 |
| Last update | 2026-09-28 |
| Contributors | 32 |
How to install Needle
bash pip install cactus-needle
Navid Moazzez
AI business strategist & AI OS builder
Navid Moazzez helps creators and founders master AI and build their own AI Operating System (AI OS) to automate their business and life.
Navid.me is reader-supported. When you buy through links on this site, I may earn an affiliate commission. Learn more.
More repo topics
More free tools
X
Bluesky
LinkedIn
Threads
Mastodon
Instagram
TikTok
Pinterest
YouTube
Amazon KDP
Apple Books
Amazon
Substack
Spotify
Apple Podcasts
Related MCP servers & CLIs
Free AI newsletterThe most actionable AI newsletter for founders
Every week, get proven AI strategies, curated tools, and step-by-step systems to grow your audience, create better content, and build a profitable creator business.
No fluff, no filler, no BS. Just five minutes each week that might level up your online business and life.
P.S. Sign up now to get free access to my ultimate AI tools guide for creators.
