Last updated: November 14, 2026
Granite 4.1 LLMON is a pair of open IBM preview models, at 3B and 8B parameters, released November 12, 2026 under Apache 2.0. They read prompts in LLMON, a markup that labels instructions and data separately, so commands hidden in documents or tool results are mostly ignored. IBM calls them experimental and not ready for production.
This guide covers what LLMON changes, IBM’s injection results, benchmark trade-offs and setup. Figures come from IBM’s announcement on Hugging Face, the 8B and 3B model cards, and IBM’s post-training paper.
Contents
- What is new in Granite 4.1 LLMON
- How LLMON works
- Prompt injection results
- General benchmarks compared
- How to run the models
- Limitations
- Frequently asked questions
What is new in Granite 4.1 LLMON
IBM released two models: ibm-granite/granite-4.1-3b-llmon-preview and ibm-granite/granite-4.1-8b-llmon-preview. Both start from Granite 4.1 base checkpoints rather than the released Granite 4.1 Instruct models, so the new prompt format is learned during post-training instead of being added on top of an existing chat format.
The target problem is indirect prompt injection: a malicious instruction hidden inside a retrieved document, a tool response or pasted user data. Ordinary instruct models often follow such text. The LLMON models are trained on a simple rule. Instructions and system prompts reached through an execution block are authoritative. Everything reached as input, including data, documents, tool schemas and tool results, is material to read, not a command to follow.
IBM also published a GitHub repository with converters, editor plugins for VS Code and Emacs, and example notebooks, plus an attention visualization demo on Hugging Face Spaces.
How LLMON works
LLMON stands for LLM Object Notation and is pronounced “lemon.” IBM describes it as an XML-like markup whose structural delimiters are reserved special tokens rather than ordinary characters. In JSON, a quotation mark inside a document and one that closes a field are the same token to a model; LLMON’s six delimiters are separate from normal text.
Every part of a prompt becomes its own block with a two-character ID: instr for instructions, data for input data and tool results, req for output requirements, doc for retrieval documents, tool for tool definitions and system for the system prompt. A final exec block then names which instruction to run, which inputs it operates on, which requirements apply and where the output goes.
Blocks that the exec block does not reference stay in the context but are left unbound. During training IBM added decoy instructions and data blocks that were deliberately left unbound, so the model learned to follow the execution block and ignore the rest. Supervised fine-tuning was followed by five reinforcement learning stages and a final preference-tuning stage built from the model’s own responses to injection attempts.
Prompt injection results
IBM’s paper reports attack success rate, the share of adversarial prompts in which the model follows the injected instruction. Lower is better. OPI places the injection in the data of classic language tasks; ASB places it in an agent’s tool observations.
| Model | OPI attack success | ASB attack success |
|---|---|---|
| Granite 4.1 3B Instruct | 100.0% | 99.2% |
| Granite 4.1 3B LLMON preview | 0.1% | 5.6% |
| Granite 4.1 8B Instruct | 96.3% | 52.8% |
| Granite 4.1 8B LLMON preview | 0.02% | 3.1% |
In the announcement, IBM says the LLMON previews avoided 47% more OPI attacks than the best frontier model it tested and 95% more than the best open-source model. IBM notes that these benchmarks measure static indirect injection, and that it is still working on adaptive attacks spread over many chat turns. All figures are IBM’s own.
If you build agents that browse, open files or operate a desktop, the same risk appears in hosted tools; our Gemini Computer Use guide covers Google’s safety controls for that tool.
General benchmarks compared
IBM’s goal was parity with the conventional instruct models, not improvement. The model cards show mostly small differences, with gains on safety evaluations and losses on some math and reasoning tests.
| Benchmark | 3B LLMON | 3B Instruct | 8B LLMON | 8B Instruct |
|---|---|---|---|---|
| MMLU (5-shot) | 68.05 | 67.02 | 74.20 | 73.84 |
| MMLU-Pro (5-shot, CoT) | 46.08 | 49.83 | 53.11 | 55.99 |
| IFEval average | 81.02 | 82.30 | 86.69 | 87.06 |
| GSM8K (8-shot) | 86.96 | 86.88 | 89.16 | 92.49 |
| HumanEval (pass@1) | 84.15 | 81.71 | 86.59 | 85.37 |
| BFCL v3 (tool calling) | 60.71 | 60.80 | 67.93 | 68.27 |
| Tulu3 safety average | 86.13 | 66.84 | 87.31 | 75.57 |
The 8B card also shows larger drops on GPQA (35.05 against 41.96), Minerva Math (72.20 against 80.10) and CRUXEval-O (36.88 against 47.63). The paper summarizes the overall capability mean as unchanged at 3B and 1.4 points lower at 8B.
How to run the models
The model cards show plain Hugging Face Transformers usage. You do not write markup by hand: each model ships a chat_template.jinja that renders ordinary messages into LLMON.
- Install
transformersand loadibm-granite/granite-4.1-8b-llmon-preview(or the 3B model) withAutoTokenizerandAutoModelForCausalLM. - Build a user message with named parts:
instruction,dataandrequirement, each a list of objects withvalueandexec. Retrieval content goes indocumentsand functions intools, using OpenAI’s function schema. - Set
"exec": Trueon the instruction and on the data it should operate on. Parts markedFalsestay in context but are referenced by nothing. - Call
tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt", return_dict=True), thenmodel.generate. - Decode with
skip_special_tokens=Falseif you want to inspect the rendered blocks. Tool calls come back as<tool_call>JSON. - For structured payloads such as JSON, install the converters from the GitHub repository with
pip install -e .to get commands such asjson-to-llmon, which keep nested fields as typed blocks.
A plain content string on a user message is treated as a single executable instruction, so existing code runs, but you only gain the separation when you label untrusted parts as data. IBM also points to its Mellea library, which applies LLMON automatically. For other small models aimed at local tool use, see our BotPsy 0.8B guide; for desktop hardware needs, see LM Studio system requirements.
Limitations
- Preview only. IBM says the models are for exploration and evaluation, not production.
- Not a guarantee. The privilege rule is learned behavior. The model card warns against treating it as the sole defense for an agent with access to sensitive systems.
- Format-dependent. Prompts in a conventional chat format are out of distribution and will degrade quality, so always use the bundled template.
- Official runtime. IBM’s cards document Transformers usage. Before relying on community conversions for llama.cpp or similar runtimes, confirm that they keep the reserved tokens and the chat template.
- No hosted pricing. The announcement and model cards list no hosted API or price; this is an open-weights release.
For another recent open model built for narrow, reliable decisions rather than open-ended chat, compare our Perplexity Decider v1.3 guide.
Frequently asked questions
What is Granite 4.1 LLMON?
It is a pair of experimental IBM models, granite-4.1-3b-llmon-preview and granite-4.1-8b-llmon-preview, released November 12, 2026 under Apache 2.0. They read prompts written in LLMON, a markup that separates instructions from data, so instructions hidden in documents or tool results are mostly ignored.
Does LLMON stop prompt injection completely?
No. IBM says the behavior is learned, not an enforced guarantee. It greatly reduces static indirect injection, but IBM is still working on adaptive attacks that unfold over many chat turns, and the model card warns against using it as the only defense for an agent with access to sensitive systems.
Do I have to write LLMON markup by hand?
No. Each model ships a chat template that turns an ordinary messages list into LLMON when you call apply_chat_template. You only label the parts of a turn, such as instruction, data and requirement, and mark each one exec True or False.
How large are the Granite 4.1 LLMON models?
The 3B model has about 3.66 billion parameters and the 8B model about 8.79 billion, both stored in bfloat16. Their config files set a maximum position length of 131,072 tokens.
Can I use Granite 4.1 LLMON in production?
IBM calls both models an experimental preview that is ready for exploration and evaluation but not for production. Use them to test the approach in agent and retrieval pipelines, and keep other safeguards in place.
Sources
- LLMON: An LLM-Native Markup Language, now in Granite 4.1 3B and 8B, IBM Granite on Hugging Face
- Granite-4.1-8B-LLMON-preview model card, Hugging Face
- Granite-4.1-3B-LLMON-preview model card, Hugging Face
- generative-computing/llmon repository, GitHub
- Post-Training Large Language Models with LLMON, arXiv