llama.cpp is a C/C++ library that runs large language models on CPUs, GPUs and even phones without installing heavy frameworks. It supports quantized models and offers an OpenAI-compatible server.
Why it matters. Developers and teams who want fast, cheap inference on their own hardware can spin up a local model without cloud costs.
Try it:curl -LsSf https://llama.app/install.sh | sh
Today in one paragraph. Today's theme is DIY AI, making everyday gadgets and content yours. From HermesEnabler2, which flips any Apple Watch into a custom Hermes face, to a Chinese GIF Factory that scripts animated stickers with a single prompt, the open-source world is handing the power back to users.
Yogurt captures your video calls and generates AI-powered notes while keeping everything on your machine. It runs locally on macOS, Windows or Linux and requires no cloud service.
Skills is a curated list of UI design and animation best-practice tips. It shows when to use a pattern, gives examples, and provides prompts you can feed to AI tools.
TensorFold is a command-line engine written in Zig that loads LLM checkpoints and generates text directly on Apple Silicon GPUs via Metal or on NVIDIA GPUs via CUDA. It performs tokenization and kernel execution itself, so you don’t need a Python runtime or ML libraries.
agentes.chat is an open-source, self-hosted team chat app that works like Slack but lets AI agents join channels as regular members. The agents can run Claude, ChatGPT, Gemini, Grok or any OpenRouter model, have their own browser, memory and are budget-tracked.
Sophon-Realtime v1 is an open-source 32-billion-parameter model that can listen and speak at the same time. It picks up tone, hesitation and laughter, then replies with matching vocal cues, offering three builds for expressive speech, speed, or low-latency output.
SWARM is a conversational AI built on Amazon Bedrock AgentCore that stores and retrieves Natera’s internal lab-operations procedures. Lab staff can ask natural-language questions and receive step-by-step guidance without leaving their workflow.
Amazon Bedrock lets you adjust four request-level settings, cache duration, service tier, reasoning effort, and maxTokens, to control how many tokens are processed and which pricing rate applies. Tweaking these options can lower your bill without changing your prompts or application logic.
Wave, Amplitude’s AI-driven product agent, scans usage data, suggests the next feature to build, writes the code, and tracks its impact. The whole loop runs on OpenAI models, letting product teams move from idea to deployment in 1-2 days.
Tag IAM users or roles and let those tags flow into the Cost and Usage Report. Then view Bedrock costs per identity in Cost Explorer and enforce limits automatically with AWS Budgets.
LLMON is a lightweight markup that tags parts of a prompt with special tokens so the model can see structure explicitly. IBM has fine-tuned the 3 billion-parameter and 8 billion-parameter Granite 4.1 models to read and produce text wrapped in this notation.
Amazon Bedrock’s blog explains a method to measure the real cost of each AI task, beyond the simple per-token price. It walks through four hidden cost factors and shows how caching, model routing and output limits can lower the bill while proving an agent’s ROI.
LiteLLM runs on AWS Fargate and forwards calls to Amazon Bedrock models through an OpenAI-style endpoint. A single API key gives developers instant access while admins can set per-user costs and quotas.
Gemini 4 Hydrogen is a large language model tuned for speed and low cost, supporting up to 1 million tokens per request. It targets everyday coding, testing and other agentic workflows and is reachable via Google AI Studio, the Gemini API, and related Google services.
Google Search now runs on the Gemini 4 Hydrogen model, delivering more helpful, visual and intuitive answers. The AI can handle complex, multi-part queries, show tables, images and link to relevant sites.
Research
The papers researchers upvoted most today, top 9 of 22
One Tab evaluates a single model’s ability to generate new rows, fill missing values, and predict outcomes from the same table. It shows which capabilities overlap and whether a generalist model can handle all three.
SteerDistill uses a teacher model to steer low-bit LLM agents back toward successful behavior. It fixes the common failure where quantized agents ignore task details or environment cues.
The paper introduces a reinforcement-learning loop that rewards a language model for pulling in external information and using it to improve its answers. This lets the model acquire knowledge on-the-fly instead of relying only on its static parameters or simple retrieval.
SparseRead is a method that lets an AI agent pull only the relevant snippets from a webpage, document, or tool output instead of loading the whole thing. By reading less, the agent uses fewer tokens and answers faster.
It lets a service pick a custom mix of language models for each prompt, adjusting the number and size of models to hit a target quality or budget. This replaces the usual one-model-per-query approach and can lower compute spend while keeping answers acceptable.
GTAGym is an open-source environment that lets reinforcement-learning agents practice and be evaluated on real-world tasks inside commercial AAA video games. It provides a gym-compatible API and automatic scoring so you no longer need human annotators to judge success.
Interactive Visual Gym is a benchmark suite that creates virtual environments with simulated users to evaluate visual interaction models in closed-loop settings. It shows how an assistant’s responses influence later user actions and overall task success.
The paper proposes a serving architecture that lets large language model agents dynamically adjust compute allocation and KV-cache handling at runtime. This flexibility aims to improve latency and resource utilization for ever-changing agent tasks.
The paper re-examines the Platonic Representation Hypothesis, which says larger models develop similar internal representations across tasks. It presents recent findings that this alignment may not keep improving as models scale.
7 upvotes
The conversation
What builders are arguing about on Hacker News, top 8 of 9
ZigZag is a desktop app that turns vector graphics into embroidery files and can talk to AI agents via its Model Context Protocol. It exports common formats and can stream designs straight to a Brother Skitch over Bluetooth.
Gemini 4 Hydrogen is a large language model optimized for speed and low cost, aimed at everyday coding and agentic tasks. It offers a 1 million-token context window and is priced at $1.50 per million input tokens and $7.50 per million output tokens.
Orchestrator is a new model selector in Google Antigravity that dynamically switches between the cheap Gemini 4 Hydrogen model and the higher-quality Gemini 4 Carbon model as a task evolves. This keeps simple steps fast and inexpensive while calling the stronger model only when needed.
smolmaxxing finds the biggest language model a low-end device can actually run and then pushes it further with custom kernels, smarter memory handling and weight streaming. It demonstrates a 0.8 B-parameter model at 16 tok/s on a cheap TV box and a 1.7 B model at 25 tok/s on a 2013 iPad.
Apple’s new method speeds up maximum inner product search, the core operation behind recommendation engines and retrieval-augmented generation. It reduces the need for exhaustive comparison by building an index that accounts for the difference between short queries and long document keys.
The article shows that U.S. workers’ 401(k)s and IRAs are being invested in companies building massive AI data centers. Even as many Americans protest these projects, their retirement money is quietly financing them.
Aircad runs 2D drafting and 3D solid modeling entirely in the browser. It adds multiplayer editing and an on-screen AI that can follow plain-language commands to draw, model, and render.
Klad is a 2-D painting and animation editor you share with a coding agent. Both you and the agent can draw using the same brushes, layers and timeline, and each stroke appears instantly for the other.