Thinking Machines Lab Releases Inkling, Their First Open-Source AI Model

Thinking Machines Lab has entered the open-source arena with the release of the Inkling model family. The startup, founded by former OpenAI CTO Mira Murati, spent most of its existence in near-total silence. Inkling is the first thing they have shipped publicly, and the decision to ship it as an open-weight model rather than a locked API says a lot about where they think the industry is heading.

The headline numbers: 975 billion total parameters, built on a Mixture-of-Experts architecture. That total sounds enormous, but the number that matters for anyone actually planning to run it is the active parameter count, and MoE is the reason those two figures diverge so sharply.

In a traditional dense model, every parameter participates in every token you generate. A 975B dense model would be effectively unrunnable outside a serious datacenter, and slow even there. Mixture-of-Experts changes the arithmetic. The network is split into many specialized sub-networks, the experts, and a routing layer decides which handful of them to activate for each token.

The practical consequence is that you get the knowledge capacity of a very large model while paying the inference cost of a much smaller one. The full parameter count determines how much the model can know. The active count determines how fast and how cheaply it answers. This is why MoE has become the default architecture for nearly every frontier open-weight release: it is the only approach that scales capacity without scaling your inference bill at the same rate.

The tradeoff is memory. You still have to hold the full parameter set somewhere, because the router might call on any expert at any moment. A 975B model demands serious VRAM or a quantized build, even though only a fraction of it fires per token. Anyone reading "MoE" as "cheap to host" is going to be disappointed the first time they check the hardware requirements.

Rather than scaling parameter counts indiscriminately, Inkling focuses on synthetic data curation, distillation from frontier reasoning traces, and highly optimized attention kernels.

Each of those three is a deliberate bet. Synthetic data curation means generating and filtering training data rather than scraping ever-larger piles of raw internet text. The open web has largely been exhausted as a resource, and what remains is increasingly polluted with AI-generated content, so the frontier has moved toward carefully constructed data instead.

Distillation from reasoning traces means training the smaller model to imitate not just the answers a larger model produces, but the intermediate steps it takes to get there. You are teaching the process, not the conclusion. This is a large part of why recent mid-size models reason so much better than their parameter counts would suggest.

Optimized attention kernels are the least glamorous item on the list and possibly the most important. Attention is the dominant cost in transformer inference, and it scales badly with context length. Improvements at the kernel level translate directly into longer usable contexts and lower latency, with no change to the model's intelligence at all.

Releasing the weights rather than an API changes what you are allowed to do with the model, and the difference is bigger than most people assume.

1. You can run it on your own hardware. Nothing leaves your infrastructure. For anyone handling regulated data, medical records, legal documents, or proprietary source code, this is frequently the only arrangement that clears a compliance review.

2. You can fine-tune it. Open weights let you specialize the model on your own domain. A model that knows your codebase or your internal terminology often outperforms a stronger general-purpose model on your actual work.

3. It cannot be deprecated out from under you. API models get retired, re-priced, and silently updated. A local model behaves exactly the same way in two years as it does today, which matters enormously if you are building a product on top of it.

4. You can inspect it. Researchers can probe the actual network rather than guessing at behavior from the outside. That is the only path to serious interpretability work.

With native support for Ollama, llama.cpp, and vLLM, Inkling integrates into existing local-first and enterprise pipelines without custom plumbing.

That list covers the three situations most developers are actually in. Ollama is the fastest way to get a model running on a laptop, and it handles the download, quantization, and serving with essentially one command. llama.cpp is the portability layer, the thing that lets a model run on consumer GPUs, Apple Silicon, and hardware that has no business running a frontier model. vLLM is the production answer, built for high-throughput batched serving with efficient memory paging.

Shipping day-one support for all three means the model is usable immediately rather than after the community spends a month writing conversion scripts. It is an unglamorous detail that determines whether an open-weight release gets adopted or quietly ignored.

If you are already running open-weight models and have the hardware, Inkling is worth benchmarking against your current setup on your own tasks. Published scores are a starting point, not a decision.

If you are choosing between this and a hosted API, the honest answer is that it depends on volume and sensitivity. Low volume with no privacy constraints usually favors an API. High volume, or data you cannot send to a third party, favors weights you control. The break-even point arrives faster than most teams expect.

And if you are new to local models entirely, start with something smaller. Get comfortable with quantization, context limits, and the practical realities of local inference on a model that fits on your machine first. Inkling will still be there.