On Thursday, PrismML released Bonsai 2 27B, its latest in a family of models, which compresses Qwen3.8 27B, a widely used open source model from Alibaba, down to 5.9 GB. That’s small enough to fit on a PC and, possibly, a high-end smartphone. It’s a 9x to 10x reduction in memory versus the original.
Kai examines the technical requirements and hidden infrastructure demands of running DeepSeek V4.1 Flash on personal hardware. The analysis covers the mixture-of-experts architecture, memory allocation challenges, and the practical trade-offs between local hosting versus API usage for developers.
Kai examines the technical shift enabling large-scale artificial intelligence models to run on consumer-grade hardware. By analyzing new architectural developments in model storage and memory tiering, this exploration breaks down the trade-offs between local inference costs and processing speeds when handling complex tasks compared to traditional cloud-based solutions.
Sam Witteveen explores the release of PAIR, an open-source tool designed to manage local AI agents across multiple household devices. The video explains how this virtual inference router enables parallel task delegation, distributing workloads across various hardware configurations to improve efficiency for agentic software workflows.
Kai examines why local AI coding agents on Mac devices often experience significant performance degradation over time due to inefficient prefix caching. The guide details how oMLX implements a two-tier caching system to address this, along with instructions for integrating the tool with popular coding environments to maintain consistent response speeds throughout long sessions.
In this video, I look at the long awaited Qwen3.8-27B model. Both what it can do and how to serve it at the maximum tokens per second
Better Stack explores how Turbo Fieldfare utilizes the unique architecture of Apple Silicon to run massive mixture-of-experts models. By streaming specific model components directly from SSD and leveraging unified memory, this approach significantly reduces RAM requirements, making advanced AI performance more accessible on local hardware.
In this video, Sam Witteveen looks at ThinkingCap, a fine-tune of the Qwen3.6-27B model. BottleCap's goal was intentionally conservative: do not try to make the model smarter or teach it new capabilities. They wanted to preserve knowledge, reasoning ability, answer quality, conversational style, instruction following, and safety behavior.
The only thing they wanted to change was the amount of computation spent reaching an answer. In other words: keeping the same model, and making sure it overthinks less.
Sam Witteveen explores the capabilities of the new AMD Ryzen AI Halo, which features 128GB of unified memory. The demonstration highlights how this architecture enables local operation of large language models, image generation tools, and fine-tuning workflows, while comparing the practical benefits of unified memory against traditional workstation setups with discrete GPUs.
Sam Witteveen explores this new family of self-scaffolding models designed to generate their own task-specific harnesses alongside solutions. By utilizing a two-stage reinforcement learning process, these models aim to optimize both the coding environment and agentic trajectories, offering a versatile approach for handling complex local coding tasks without requiring human-authored scaffolds.
Run AI agents locally with privacy, low latency, and offline access. Dive into how MLX advancements and Mac hardware make powerful agentic workflows possible entirely on-device. You’ll explore code agents such as OpenCode, see how they integrate into Xcode, learn techniques for multi-Mac scaling, and discover how to integrate tools seamlessly — without ever leaving your machine.
On-device AI models have stayed small because the entire weight set has to live in DRAM, capping practical parameter counts well below what server-side deployments use. Enterprise architects evaluating agentic workloads have had to choose between capable cloud-dependent models and limited on-device ones. Apple's third-generation foundation models, announced at WWDC26, break that constraint by moving the weight set off DRAM entirely.
While many AI open source model providers are pursuing larger and more powerful models, Google is still giving attention to the smaller, more local side of the market. Today, the tech giant released Gemma 4 12B, an 11.95-billion-parameter open-weights model with permissive Apache 2.0 license optimized to execute locally on a standard enterprise laptop using just 16GB of VRAM or unified memory.
That means those enterprise users looking to keep working with AI while on a flight without WiFi, or trying to keep it offline for security reasons, can now do so far more easily and at far less cost (free to download and operate).
In this video, we look at running local AI work jobs for LLMs, images and video models, but running it on an AMD GPUs and processors.
What if you could get a Claude Code-level AI coding assistant for FREE running locally on your machine? In this video, I show you how to combine Gemma 4 with Ollama to build a powerful, privacy-first, local AI coding setup that behaves surprisingly close to Claude Code.
In this video I walk through the full setup of Hermes Agent from scratch. We plug it into Gemma 4 running locally through Ollama and set up self-hosted Firecrawl for private web search. By the end, you have a fully local, fully private AI agent connected to Telegram with no paid APIs required.
Ollama, a runtime system for operating large language models on a local computer, has introduced support for Apple’s open source MLX framework for machine learning. Additionally, Ollama says it has improved caching performance and now supports Nvidia’s NVFP4 format for model compression, making for much more efficient memory usage in certain models.
In this video, we look at LiteParse, a new open document parser created by the people at LlamaIndex. This library allows you to pass a variety of different types of documents. and output easily to text files or JSON.
In this video, I look at using Claude Code with Ollama's new function called Ollama Launch along with the GLM 4.7 Flash model.
Learn how to run powerful AI models locally on your computer for FREE - achieving 80% of ChatGPT's capabilities without sending any data to external servers. Perfect for companies with privacy restrictions or anyone wanting to save on AI subscription costs.
