NVIDIA Nemotron Bolt: Can AI agents run locally in 4 GB?
NVIDIA's Nemotron 3.5 Bolt E4B targets local AI agents. What 4 GB, 64K context and tool-use benchmarks mean, and what remains unverified.
NVIDIA released the open-weight Nemotron 3.5 Bolt E4B 4GB model on October 23, 2026. In specified quantized configurations, NVIDIA reports GPU memory use around four gigabytes while supporting multi-turn tool use for local conversational and device agents. That number is not a universal hardware requirement, and memory can increase as context or application complexity grows.
CLAIM-LEVEL EVIDENCE
Source evidence and remaining unknowns
- Primary source – Release:
Release:
NVIDIA announced downloadable Bolt E4B checkpoints on October 23.
- Vendor claim – Memory and context:
Memory and context:
Roughly 4-GB configurations and a 64K context option are supported claims under different operating conditions.
- Vendor claim – Benchmark gains:
Benchmark gains:
The approximately 40% average improvement and up-to-80% token reduction come from NVIDIA's selected comparisons.
- Open question – Operational readiness:
Operational readiness:
Reliable end-to-end tool use, target hardware, security and full deployment licensing have not been independently tested here.
Vendor and government statements do not establish independent operational performance.
Why a smaller agent model matters
A useful agent must do more than answer a prompt. It may select functions, interpret tool results, maintain a conversation across several turns and decide when not to act. Large hosted models are capable, but device assistants, robotics interfaces, interactive game characters and industrial systems have their own constraints: latency, limited hardware budgets and controls over data leaving the device.
Nemotron 3.5 Bolt E4B is NVIDIA's attempt to optimize for those conditions. It is based on Gemma 4 E4B and further trained for conversation, instruction following and multistep tool use. NVIDIA lists a 64K-token context option, configurable reasoning and downloadable checkpoints, including a quantized GGUF version and a BF16 release suitable for research and customization. Availability of weights does not remove the need to check license terms and downstream component requirements.
Does it really fit in 4 GB?
The critical distinction is between a model profile and an entire running assistant. In one published NVIDIA measurement on an RTX 5070, a short prompt-and-answer run used approximately 3.5 GB of VRAM. A different scenario with a larger cache consumed roughly 4.3 GB. NVIDIA's Jetson Orin Nano measurements range higher under the configurations shown.
Longer histories and larger KV caches can increase memory needs. Speech recognition, text-to-speech, vision processing, tool runtimes and the operating environment add their own requirements. The headline should therefore be interpreted as evidence of an efficient specific inference configuration, not a guarantee that a complex offline assistant fits on any 4-GB device.
The practical value of multistep tool use
NVIDIA reports roughly 40% higher average quality than its predecessor across a selected group of conversational-agent benchmarks, and up to 80% fewer generated output tokens in certain reasoning-enabled comparisons. Those are vendor measurements. The published tables also show individual categories where competing models outperform Bolt E4B. No single synthetic score demonstrates safe performance in a production workflow.
For a company evaluating a local agent, the meaningful questions concern correct tool selection, arguments, retries, permission boundaries and whether the system can stop or ask for clarification when a result is ambiguous. A fast response that acts on the wrong customer record is not a useful win.
How to evaluate it responsibly
An evaluation should compare the same tasks, prompts, data and allowed tools across candidate models. Include both short requests and longer workflows with changing state. Measure memory peaks, response latency, action success, manual intervention and recovery after a failed tool call. Verify that the model is allowed to run under the intended license, and assess privacy for the full application—not just the local model process.
AgentenCode assessment: This release matters because it makes locally operated, open-weight conversational agents more plausible for constrained devices. It does not establish that every device can host an independent, reliable AI agent. The path from compact model to safe product remains an engineering task.
Original sources and related reading
Related AgentenCode evidence and agent profiles →
Vendor and government statements are not independent end-to-end testing.
Frequently asked questions
Can it run without the cloud?
Locally downloaded weights enable offline inference, but optional external tools and connected services may still use the network.
Will every 4-GB device run it well?
No. Hardware, quantization, context and additional services determine actual feasibility.
Is it faster or more capable than cloud frontier models?
The provided vendor measurements cannot establish a universal answer. Test the real workload and compare cost, latency and errors.