Hy4 Open Weights: Power-User Guide to Self-Hosting, GGUF and License

Last updated: November 20, 2026

Hy4 open weights, released by Tencent on November 20, 2026, let you download and self-host Tencent’s 770B-parameter multimodal flagship model. The release includes BF16, FP8 and base checkpoints, a 1M-token context window, GGUF builds and a system card, under the Tencent Hy License with limits for very large commercial services.

Contents

What Tencent released

Tencent first launched Hy4 as an API and app model on November 4, which we covered in our Tencent Hy4 guide to vision, computer use and API pricing. At that point the final weights were still pending. On November 20 (10:52 p.m. ET on November 19), the Tencent Hy account announced on X that the open weights were out, together with the Hy4 System Card and a safety evaluation report.

According to the Hugging Face model card, three checkpoints are available on Hugging Face, ModelScope and GitCode:

  • Hy4 (tencent/Hy4): the instruct model in BF16.
  • Hy4-FP8 (tencent/Hy4-FP8): an FP8-quantized instruct model.
  • Hy4 base (tencent/Hy4-base): the pre-trained base model, not tuned for chat.

The card also links a fine-tuning pipeline and a GGUF build from Tencent’s AngelSlim compression team.

Hy4 specifications at a glance

The figures below come from the model card specification table:

  • Architecture: mixture of experts with 78 layers (1 dense, 77 MoE), 256 routed experts and 1 shared expert per MoE layer, 8 routed experts active per token.
  • Parameters: 770B total, excluding the vision encoder, adapter and drafter; 49B text plus 1.23B vision parameters active per token.
  • Context: 1,048,576 tokens.
  • Inputs: text, image and video understanding.
  • Attention: Gated DeepSeek Sparse Attention on Multi-head Latent Attention.
  • DFly2 drafter: a built-in 3.3B, five-layer speculative decoding model that reuses Hy4’s KV cache, so it needs no extra KV memory.

Which checkpoint to download

Storage and memory needs differ widely. This table combines the vLLM recipe and the AngelSlim GGUF card:

Option Size Hardware guidance Best for
Hy4-FP8 (vLLM, SGLang) 786 GB on disk, 245 shards About 944 GB total GPU memory on one 8-GPU NVIDIA node Production serving; the default in the recipe
Hy4 BF16 (vLLM) 1,549 GB on disk About 1,859 GB GPU memory (8x B300) Full-precision evaluation and fine-tuning
GGUF Q4_K_M 437.93 GiB Fits one 8x H20 node (8 x 96 GiB) llama.cpp users who want the safe default
GGUF IQ2 309.58 GiB Same node; lower -ngl with less VRAM Tighter memory budgets
GGUF STQ1_0 214.65 GiB Same node; needs an extra patch The smallest build, about 2.39 bits per weight

None of these options runs on a single consumer GPU. For a cheaper hosted option, see our Hy4 flash guide.

How to serve Hy4 with vLLM

These steps follow the vLLM recipe and the model card launch command:

  1. Provision one 8-GPU NVIDIA node with enough GPU memory for your checkpoint and enough host RAM and disk to load it.
  2. Pull the prebuilt image vllm/vllm-openai:hy4. Release v0.34.0 does not register the Hy4 architecture; the recipe says 0.35.0 will be the first release that does, and a nightly wheel also works.
  3. Download tencent/Hy4-FP8 from Hugging Face into your cache.
  4. Start the server with --tensor-parallel-size 8, --reasoning-parser hy_v4, --tool-call-parser hy_v4, --enable-auto-tool-choice, --attention-backend FLASHMLA_SPARSE and --speculative-config '{"num_speculative_tokens":5,"method":"dfly"}' to turn on the DFly2 drafter.
  5. Set --max-model-len to match your memory. The recipe starts at 131,072 tokens with 32 concurrent sequences.
  6. Point any OpenAI-compatible client at http://localhost:8000/v1 and use the recommended temperature=0.9 and top_p=1.0.
  7. For a direct answer without reasoning, send extra_body={"chat_template_kwargs": {"reasoning_effort": "no_think"}}. The default is high.

SGLang users can run the prebuilt lmsysorg/sglang:hy4 image with --tp 8 and --speculative-algorithm DFLY, per the model card. DFly2 requires single-node tensor parallelism, so the recipe does not cover multi-node or pipeline parallel setups. For recent vLLM changes, see our vLLM 0.33 guide.

Running the GGUF builds

The GGUF card says the files need a llama.cpp fork at the head of pull request #32517, plus a separate patch for STQ1_0. Key points:

  • Use --jinja. Hy4’s chat template matches no built-in llama.cpp family.
  • Leave room for reasoning. The model thinks before answering, so set a generous token limit.
  • Images: load mmproj-HY4-VL-BF16.gguf with a vision GGUF; one image uses 4 to 4,096 tokens.
  • Network storage: copy files to local disk; memory-mapped reads over NFS can be very slow.

For the stable llama.cpp server features these builds sit on, see our llama.cpp 1.0.0 guide.

License terms that matter

Hy4 uses the Tencent Hy License, Version 1.0. It is derived from Apache 2.0, and the text states it is not the Apache License.

Section 10 adds commercial terms for model-as-a-service (MaaS) operators, meaning anyone who gives third parties general remote access to inference or fine-tuning, such as through an API. A separate written license from Tencent is required if either condition applies:

  • worldwide gross revenue above US$50 million in any 12-month period, or
  • a product or service with more than 100 million monthly active users in any month.

The rule does not apply to internal use when neither the model nor its outputs are offered to third parties. Licensing questions go to hy_partners@tencent.com.

Self-hosting vs the Hy4 API

Tencent has offered Hy4 through its API since November 4. Self-hosting adds data control, fine-tuning and no per-token API fees, at the cost of node-class hardware. For most teams the hosted API is simpler; our Hugging Face OpenRouter routing guide covers one way to reach hosted open models.

In Tencent’s own finance evaluation, reported on the model card, 213 professionals scored 579 tasks. Hy4 averaged 3.04 out of 4, against 2.97 for DeepSeek-V4.1-Flash-1017 and 2.68 each for Kimi K3.1 Pro and GLM-5.3. These are vendor-run results.

Limitations to plan for

  • Gap to closed models: Tencent says a notable gap remains against top proprietary models in research-level math, long-horizon coding and advanced visual question answering.
  • NVIDIA only in vLLM: the ROCm port lacks the vision encoder and DFly2 drafter.
  • 1M context untested in the recipe: start at 131,072 tokens and raise carefully.
  • Base model: Hy4 base is not instruction-tuned and may ignore few-shot answer formats.
  • License: check Section 10 before offering a public API.

Frequently asked questions

Are Hy4 weights open source?

Tencent released the Hy4, Hy4-FP8 and Hy4 base weights on Hugging Face, ModelScope and GitCode on November 20, 2026. They use the Tencent Hy License, which is derived from Apache 2.0 but adds commercial terms for large model-as-a-service businesses.

What hardware do I need to run Hy4?

The vLLM recipe calls for one 8-GPU NVIDIA node. The FP8 checkpoint needs about 944 GB of total GPU memory and the BF16 checkpoint about 1,859 GB. AMD Instinct GPUs are not supported in vLLM for this model.

Is there a Hy4 GGUF for llama.cpp?

Yes. Tencent’s AngelSlim team publishes Hy4 GGUF files at 437.93 GiB (Q4_K_M), 309.58 GiB (IQ2) and 214.65 GiB (STQ1_0), plus a 2.49 GiB vision file. They need a patched llama.cpp build and the –jinja flag.

Can I use Hy4 commercially?

Yes, with conditions. A model-as-a-service business needs a separate written license from Tencent if its worldwide revenue exceeds US$50 million in 12 months or its product has more than 100 million monthly active users. Internal-only use is exempt.

What is the Hy4 context window?

The model card lists 1,048,576 tokens. The vLLM recipe starts at 131,072 tokens, the evaluated window, and says a full 1M-token request has not been validated.

How do I turn off Hy4 reasoning?

Pass reasoning_effort set to no_think through chat_template_kwargs in extra_body. The default, high, reasons at length before answering and suits math and coding.

Sources

Image: Tencent Hy logo from the Tencent-Hunyuan GitHub repository, Apache License 2.0, via Wikimedia Commons, resized. Tencent is a trademark of Tencent Holdings Ltd. FreeAnswers is not affiliated with or endorsed by Tencent.

Related answers

Leave a Comment