Last updated: November 15, 2026
Inkling 2 Small’s full weights went live on Hugging Face on November 12, 2026 under Apache 2.0. You can now self-host the 594B-parameter, 17B-active multimodal model with SGLang, vLLM, TokenSpeed or Unsloth, fine-tune it outside Tinker, or call it through seven API providers, including Cloudflare and Together.
This guide covers what changed since the Tinker-only launch, the hardware you need, and how to serve the model. For the launch details and benchmarks, see our earlier Inkling 2 Small guide to Tinker access and specs.
Contents
- What changed on November 12
- Checkpoints and hardware
- How to serve it with vLLM or SGLang
- Hosted APIs and pricing
- Local GGUF option and effort settings
- Limitations and comparisons
- Frequently asked questions
What changed on November 12
When Thinking Machines Lab introduced Inkling 2 Small on November 5, the model was available only through Tinker with closed weights. The company said it would release the weights under Apache 2.0 after external safety testing and red-teaming. A November 12 update at the top of the same announcement confirms the weights are downloadable, and the Hugging Face repository lists the Apache 2.0 license.
The model itself is unchanged. What changed is where and how you can run it.
| Item | November 5 launch | Since November 12 |
|---|---|---|
| Weights | Closed, Tinker only | Downloadable on Hugging Face, Apache 2.0 |
| Inference | Tinker Playground and Tinker API | SGLang, vLLM, TokenSpeed, Unsloth, plus Tinker |
| Post-training | Tinker fine-tuning | Also Miles, SkyRL and Snowflake Cortex Training |
| Hosted APIs | Tinker | NVIDIA, Fireworks, Together, Baseten, Nebius, Modal, Cloudflare |
| Checkpoint downloads | Promised with the weights release | Available, so Tinker fine-tunes can leave the platform |
The core specifications from the model card still apply: a 48-layer decoder-only Mixture-of-Experts transformer that routes each token to 8 of 384 experts, 594B total and 17B active parameters, text and image input, text output, and a context window of up to 1M tokens.
Checkpoints and hardware
Thinking Machines ships two checkpoint formats. The model was pretrained, midtrained and post-trained in NVFP4, and the company recommends the NVFP4 checkpoint for all use cases. It says the MXFP8 version gains only 1 to 2 percentage points on evaluations.
| Checkpoint | Repository | Approx. size | Minimum aggregated VRAM | Example GPUs |
|---|---|---|---|---|
| NVFP4 (recommended) | thinkingmachines/Inkling-2-Small | About 394 GB | 500GB | 2x B300 or 4x H200 |
| MXFP8 | thinkingmachines/Inkling-2-Small-MXFP8 | About 623 GB | 800GB | 4x B300 or 8x H200 |
Sizes come from the SGLang cookbook, which notes that the main checkpoint stores routed experts in NVFP4 for layers 0 to 39 and MXFP8 for layers 40 to 47. VRAM figures come from the model card. Only 17B parameters are active per token, but all 594B must sit in memory.
How to serve it with vLLM or SGLang
Neither framework had Inkling 2 Small support in a tagged release at the time of writing, so both point to dedicated images. Here is the vLLM path from the official vLLM recipe:
- Pull the
vllm/vllm-openai:inkling-2-smallimage, which the recipe says to use until support lands in a vLLM release. - Launch the server:
vllm serve thinkingmachines/Inkling-2-Small --tensor-parallel-size 4 --max-model-len 262144 --reasoning-parser inkling. Add the recipe’s DSpark speculative config to enable the bundled speculator, which drafts 7 tokens per step. - Stay at 262,144 tokens on one four-GPU tray (TP4). For the full 1,048,576-token context, run two trays at TP8 and launch every node with
--no-numa-bind. - Opt in to tool calling with the
inklingtool-call parser. The recipe notes the parser is unit-tested but no end-to-end tool-calling benchmark has been run yet. - Call it through the OpenAI-compatible endpoint at
http://localhost:8000/v1with modelthinkingmachines/Inkling-2-Small, passing images asimage_urlcontent parts.
For SGLang, pull lmsysorg/sglang:dev-inkling-2-small (separate tags exist for Vera Rubin, AMD MI350X/MI355X and TPU7x). The cookbook says every recipe enables the inkling reasoning parser and a tool-call parser named inkling2, and both its Balanced and DSpark recipes serve the full 1M context. Keep expert parallelism equal to tensor parallelism, because the NVFP4 experts cannot be TP-sharded. Our SGLang 0.5.23 guide covers the framework’s recent features.
Fine-tuning note: vLLM supports LoRA adapters, but with the NVFP4/MXFP8 experts in this checkpoint, the recipe says to use attention-only adapters.
Hosted APIs and pricing
If you do not have a multi-GPU node, a hosted endpoint is the practical route. Two providers publish clear prices:
| Provider | Model ID | Input / output per 1M tokens | Notes |
|---|---|---|---|
| Cloudflare Workers AI | @cf/thinkingmachines/inkling-2-small | $0.30 / $1.00 ($0.10 cached input) | Workers Paid plan or prepaid AI Gateway credits; 1,048,576-token context; zero data retention |
| Together AI | Inkling 2 Small | $0.30 / $1.00 | Serverless and dedicated options |
Prices are from the Cloudflare model page and Together’s model page. Cloudflare exposes reasoning levels of none, minimal, low, medium, high (default) and max, along with function calling and vision.
Local GGUF option and effort settings
Unsloth’s guide offers GGUF quantizations for llama.cpp and Unsloth Desktop. The smallest, UD-IQ1_S, needs 171.7GB of disk for weights alone; UD-IQ2_XXS needs 190.6GB and UD-Q4_K_XL needs 354.2GB. Image input needs the 658 MB mmproj-F16.gguf projector, and llama.cpp needs a build with Inkling 2 support. See our llama.cpp 1.0 guide for the runtime.
Recommended starting settings from Unsloth: temperature 1.0, top-p 1.0, top-k 0, min-p 0.0, a 32K context to start, and the embedded chat template (--jinja). Thinking Machines runs its own evaluations at temperature 1.0 and effort 0.99 (xhigh).
Effort is a number. Unsloth maps none to 0.0, minimal to 0.1, low to 0.2, medium to 0.7, high to 0.9 and xhigh/max to 0.99; the GGUF template defaults to high. A practical pattern is low effort for quick edits and summaries, high for routine agent turns, and xhigh for hard debugging or long research runs.
Limitations and comparisons
Thinking Machines lists several limitations: the model is not the most succinct communicator, does not always pick the best tool in codebases with many skills, is less optimized for chat presentation, and does not accept audio. The model card adds the usual cautions about hallucination and recommends human review for high-stakes use.
On capability, the announcement reports an Artificial Analysis Intelligence Index score of 31, 72.1% on DeepSWE v1.1 and 86.3% on SWEBench Pro v2 (Hard). In the same table, DeepSeek-V4.1-Flash-1017 scored higher on both coding tests. Inkling 2 Small’s distinguishing points are its Apache 2.0 license, native image input and fine-tuning support on several platforms.
Other recent open-weight releases worth comparing include Nex-N2.6, Edge0-145B and Meta’s Muse Spark Open. Check each model card for hardware and license terms before choosing one.
Frequently asked questions
When were the Inkling 2 Small weights released?
Thinking Machines Lab launched Inkling 2 Small on Tinker with closed weights on November 5, 2026. An update dated November 12, 2026 on its announcement says the full weights are now available for download on Hugging Face.
What license does Inkling 2 Small use?
The Hugging Face repository lists the Apache 2.0 license, the license Thinking Machines said it would use when it announced the model. Apache 2.0 permits commercial use, modification and redistribution.
How much GPU memory does Inkling 2 Small need?
The model card says the main NVFP4 checkpoint needs at least 500GB of aggregated VRAM, such as 2x NVIDIA B300 or 4x H200. The MXFP8 checkpoint needs at least 800GB, such as 4x B300 or 8x H200.
Can I run Inkling 2 Small on a single consumer GPU?
Not at full precision. Unsloth publishes GGUF quantizations for llama.cpp, but its smallest 1-bit quant still needs 171.7GB of disk space for the weights alone, so you need very large combined RAM and VRAM or unified memory.
Which APIs serve Inkling 2 Small?
Thinking Machines lists NVIDIA, Fireworks, Together, Baseten, Nebius, Modal and Cloudflare, plus its own Tinker platform. Cloudflare Workers AI and Together both list $0.30 per million input tokens and $1.00 per million output tokens.
Does Inkling 2 Small support audio or video input?
No audio. The model card lists text and image input with text output, and the announcement says this model does not support audio. Images work best with each dimension between 40 and 4,096 pixels.
Sources
- Introducing Inkling 2 Small, Thinking Machines Lab (November 5, 2026, updated November 12, 2026)
- thinkingmachines/Inkling-2-Small model card, Hugging Face
- thinkingmachines/Inkling-2-Small recipe, vLLM Recipes
- Inkling-2-Small deployment cookbook, SGLang Documentation
- inkling-2-small model page, Cloudflare Workers AI docs