Qwen3.8-27B GGUF: Unsloth Dynamic 3.1 Quants Setup Guide

Last updated: November 15, 2026

Unsloth replaced most Qwen3.8-27B GGUF files on November 10 and 11, 2026 with Dynamic 3.1 quants, which it says are 5 to 10 percent more accurate at the same size. The recommended UD-Q4_K_XL file is 17.7 GB and runs in about 16 to 19 GB of combined RAM and VRAM with llama.cpp or Unsloth Desktop.

This guide explains which files changed, how to pick a quant for your hardware, and how to run and tune the model locally. Qwen3.8-27B is a 27B-parameter dense model from the Qwen team with image and video input, hybrid thinking and a 262,144-token native context.

Contents

What changed with Dynamic 3.1
Which files were updated
Choosing a quant for your hardware
How to run it with llama.cpp
Settings, thinking effort and agents
Limitations and alternatives
Frequently asked questions

What changed with Dynamic 3.1

Unsloth’s Qwen3.8 guide carries a November 11 update: the Qwen3.8-27B GGUFs now use Unsloth Dynamic V3.1, a preview of its updated quantization algorithm. The Dynamic 3.1 page says the same method was applied earlier to DeepSeek-V4.1-Flash-1017 and Qwen3.8-Flash-Next.

Unsloth reports that V3.1 improves average accuracy by 5 to 10 percent at comparable sizes. Its testing covered agentic benchmarks such as Terminal Bench 2.1 and DeepSWE v1.1, plus LiveCodeBench v6, GPQA Diamond, MMLU-Pro and AIME25. It ran the agentic tests at 256K context and the rest at 128K. These are vendor-run results, so treat them as a guide rather than a guarantee for your workload.

The quants also keep features Unsloth added at launch: developer role support for agent tools such as Codex, MTP (multi-token prediction) for faster inference, and improved parsing of nested objects in tool calls.

Which files were updated

The Hugging Face repository history shows three Dynamic 3.1 uploads. The first, on November 10, replaced UD-IQ2_XXS. The second, on November 11, replaced twelve more files. A third commit late on November 11 (Eastern Time) replaced UD-Q3_K_XL.

That last change is worth noting. Unsloth’s guide says 3-bit quants performed on par with the older version and were not updated, and a commenter in the Dynamic 3.1 announcement thread repeated that. The file history, however, shows a separate “New UD3.1 UD-Q3_K_XL” commit. The IQ3 files were not touched.

Quant	File size	Dynamic 3.1?
UD-IQ2_XXS / UD-IQ2_XS / UD-IQ2_S	7.21 / 7.63 / 8.42 GB	Yes
UD-Q2_K_XL	9.95 GB	Yes
UD-Q3_K_XL	13.1 GB	Yes (separate commit)
UD-IQ3_XXS / UD-IQ3_S	10.9 / 12.0 GB	No
UD-IQ4_XS	14.4 GB	Yes
UD-Q4_K_M / UD-Q4_K_XL	16.6 / 17.7 GB	Yes
UD-Q4_K_S, Q4_0, Q4_1	15.4 / 16.1 / 17.5 GB	No
UD-Q5_K_S / UD-Q5_K_M / UD-Q5_K_XL	18.8 / 19.9 / 21.0 GB	Yes
UD-Q6_K / UD-Q6_K_M / UD-Q6_K_L	22.0 / 23.1 / 24.2 GB	Yes
UD-Q6_K_XL, UD-Q8 files, Q8_0, UD-IQ1	6.19 to 31.5 GB	No

If you downloaded an updated quant before November 10, download it again to get the new weights.

Choosing a quant for your hardware

Unsloth’s hardware table counts total memory, meaning RAM plus VRAM or unified memory on a Mac:

Precision	1-bit	2-bit	3-bit	4-bit	6-bit	8-bit	BF16
Total memory	7-8 GB	9-11 GB	12-14 GB	16-19 GB	23-26 GB	31 GB	56 GB

Unsloth says 4-bit quants fit GPUs such as the RTX 5080 or RTX 4090, or a Mac with 24 GB of RAM. Plan an extra 1 to 2 GB if you enable MTP. Image input also needs the vision projector file (mmproj-F16.gguf or mmproj-BF16.gguf, about 0.93 GB each).

If you have an NVIDIA Blackwell GPU, the separate unsloth/Qwen3.8-27B-NVFP4 checkpoint is an alternative to GGUF. Unsloth says it runs about 1.5 times faster than BF16 at a comparable file size, and its guide gives serving commands for vLLM and SGLang.

How to run it with llama.cpp

Build the latest llama.cpp from GitHub with CUDA on (-DGGML_CUDA=ON), or with it off for CPU or Apple Metal. Our llama.cpp 1.0.0 guide covers the current release.
Install the Hugging Face CLI with pip install -U "huggingface_hub[cli]".
Download one quant: hf download unsloth/Qwen3.8-27B-GGUF --local-dir unsloth/Qwen3.8-27B-GGUF --include "*UD-Q4_K_XL*". Change the pattern to pick a different size.
Start a chat: ./llama.cpp/llama-cli --model unsloth/Qwen3.8-27B-GGUF/Qwen3.8-27B-UD-Q4_K_XL.gguf --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0.
To serve an API instead, use llama-server with the same model and sampling flags, or let Unsloth do it with unsloth run --model unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_XL.

Unsloth Desktop, available for macOS, Windows and Linux, is the no-terminal option. Search for Qwen3.8, choose a quant, and it sets the inference parameters for you and offloads to RAM automatically.

Settings, thinking effort and agents

Qwen3.8-27B is a hybrid thinking model, and the recommended settings differ by mode:

Parameter	Thinking mode	Instruct (non-thinking) mode
temperature	1.0	0.7
top_p	0.95	0.80
top_k	20	20
min_p	0.0	0.0
presence_penalty	0.0	1.5

Thinking effort defaults to xhigh. To shorten reasoning, pass --chat-template-kwargs '{"reasoning_effort":"medium"}' to unsloth run or llama-server, and swap in low or none as needed. On Windows PowerShell, escape the inner quotes. The model also supports preserve_thinking, which keeps earlier reasoning in context; it uses more tokens but can help accuracy in long conversations.

For coding agents, the Unsloth model page lists one-line launchers such as unsloth start claude, unsloth start codex and unsloth start opencode, each followed by --model unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_XL. If you prefer Qwen’s own agent, see our Qwen Code 1.0 Preview guide.

Limitations and alternatives

Dynamic 3.1 is labeled a preview, and Unsloth says an improved version is planned for Qwen4.
Accuracy gains are Unsloth’s own measurements. Unsloth also disputes some third-party quant comparisons, so run your own tests on your workload.
Native context is 262,144 tokens. Going toward 1,000,000 tokens requires RoPE scaling such as YaRN and much more memory.
Compared with the earlier Qwen3.6-27B, the benchmark table in Unsloth’s guide shows Qwen3.8-27B scoring 73.0 versus 63.4 on Terminal Bench 2.1 and 61.7 versus 53.5 on SWE-bench Pro.

If you need a larger local model, compare our guides to Edge0-145B and Inkling 2 Small open weights, both of which need far more memory than a 27B dense model.

Frequently asked questions

What is Unsloth Dynamic 3.1?

Dynamic 3.1 is a preview version of Unsloth's quantization method for GGUF files. Unsloth says it improves average accuracy by 5 to 10 percent at comparable file sizes on most benchmarks, and it applied it to Qwen3.8-27B GGUFs on November 11, 2026.

Which Qwen3.8-27B GGUF should I download?

Unsloth recommends UD-Q4_K_XL, a 17.7 GB file that needs about 16 to 19 GB of combined RAM and VRAM. Use a 2-bit quant such as UD-Q2_K_XL (9.95 GB) on smaller machines, or a 5-bit or 6-bit quant if you have memory to spare.

Do I need to re-download my Qwen3.8-27B GGUF?

Only if you use an updated file. All UD 2-bit files, UD-IQ4_XS, UD-Q4_K_M, UD-Q4_K_XL, 5-bit and most 6-bit files were replaced, and the repository history shows UD-Q3_K_XL was also replaced. Q4_0, Q4_1, Q8_0, UD-Q4_K_S, the IQ3 files and the 8-bit UD files were not.

How much memory does Qwen3.8-27B need at 4-bit?

Unsloth lists 16 to 19 GB of total memory (RAM plus VRAM, or unified memory) for 4-bit quants, such as an RTX 5080, RTX 4090 or a Mac with 24 GB of RAM. Add 1 to 2 GB of headroom if you use MTP for faster inference.

What context length does Qwen3.8-27B support?

The model card lists 262,144 tokens natively, extensible up to 1,000,000 tokens with RoPE scaling such as YaRN. Longer contexts need more memory for the KV cache, so start smaller on consumer hardware.

How do I turn down thinking in Qwen3.8-27B?

Thinking is on by default at xhigh effort. In unsloth run or llama-server, pass –chat-template-kwargs with a reasoning_effort value of medium, low or none to shorten or skip reasoning traces.

Qwen3.8-27B GGUF: Power-User Guide to Unsloth Dynamic 3.1 Quants