HowdoyougetreliableJSONfromOllamawithstructuredoutputs?

How do you request structured output from Ollama?

You request structured output from Ollama by setting the format field on /api/chat (or /api/generate) to either the string "json" or a full JSON schema. The two do different jobs, and most people who think Ollama structured output is unreliable are using the first one.

The Ollama docs show the schema form against the native API:

curl -X POST http://localhost:11434/api/chat -H "Content-Type: application/json" -d '{
  "model": "gpt-oss",
  "messages": [{"role": "user", "content": "Tell me about Canada."}],
  "stream": false,
  "format": {
    "type": "object",
    "properties": {
      "name": {"type": "string"},
      "capital": {"type": "string"},
      "languages": {"type": "array", "items": {"type": "string"}}
    },
    "required": ["name", "capital", "languages"]
  }
}'

The JSON arrives as a string in message.content. You still have to parse it. Two settings from the same docs page make it more dependable:

• Put the schema in the prompt as well. The docs say it is “ideal to also pass the JSON schema as a string in the prompt to ground the model’s response.” The reason is mechanical and covered below: the model cannot see format.
• Lower the temperature, for example options: {"temperature": 0}, for more deterministic output.

Schema support arrived with the structured outputs release in December 2024. If format with an object does nothing on your machine, check the version first.

How do you use Ollama structured output from Python and JavaScript?

You define the shape once with Pydantic or Zod, send its JSON schema as format, and validate the reply against the same object. That gives you one source of truth for both the constraint and the check.

Python with Pydantic (from the official docs, pip install -U ollama pydantic):

from ollama import chat
from pydantic import BaseModel

class Country(BaseModel):
    name: str
    capital: str
    languages: list[str]

response = chat(
    model="gpt-oss",
    messages=[{"role": "user", "content": "Tell me about Canada."}],
    format=Country.model_json_schema(),
    options={"temperature": 0},
)

country = Country.model_validate_json(response.message.content)

JavaScript with Zod (npm i ollama zod). The current docs use Zod’s built-in z.toJSONSchema(); the original 2024 blog post used the separate zod-to-json-schema package, which you will still see in older tutorials:

import ollama from 'ollama'
import * as z from 'zod'

const Country = z.object({
  name: z.string(),
  capital: z.string(),
  languages: z.array(z.string()),
})

const response = await ollama.chat({
  model: 'gpt-oss',
  messages: [{ role: 'user', content: 'Tell me about Canada.' }],
  format: z.toJSONSchema(Country),
})

const country = Country.parse(JSON.parse(response.message.content))

Nested models work the same way. The docs’ extraction example wraps a Pet model in PetList(pets: list[Pet]) and fills it from a paragraph of free text, which is the pattern most real pipelines use: one top-level object, a list inside it, optional fields typed as str | None.

Keep the validation step even though decoding is constrained. Constrained decoding controls shape, and Pydantic or Zod is where you catch a value that is the right type but wrong, plus the truncation case in the troubleshooting table.

Does Ollama structured output work through the OpenAI-compatible API?

Yes. Ollama’s /v1/chat/completions accepts response_format, so the OpenAI SDKs can request schema-constrained JSON from a local model. The OpenAI compatibility page lists JSON mode and response_format as supported, and the structured outputs docs say the feature “works through the OpenAI-compatible API via response_format”.

The Ollama blog shows the Python SDK’s parse helper pointed at the local server:

from openai import OpenAI
from pydantic import BaseModel

class Country(BaseModel):
    name: str
    capital: str
    languages: list[str]

client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")

completion = client.beta.chat.completions.parse(
    model="gpt-oss",
    messages=[{"role": "user", "content": "Tell me about Canada."}],
    response_format=Country,
    temperature=0,
)
country = completion.choices[0].message.parsed

Two limits apply. The compatibility page does not list response_format for /v1/responses, so use chat completions for structured output. And /v1 cannot set context length per request, which matters when you extract from long documents; the Ollama API guide covers the Modelfile and environment-variable workarounds.

How does constrained decoding work under the hood?

Ollama converts your JSON schema into a grammar and, at every generation step, removes the tokens that would violate it before one is sampled. The model’s probabilities are left alone for every token the grammar allows.

The approach comes from llama.cpp’s GBNF grammars. Per the llama.cpp grammar docs, GBNF is a BNF-style format that can “force the model to generate valid JSON”, and llama.cpp converts a subset of JSON Schema into GBNF automatically. Developer Daniel Clayton traced the path in Ollama’s code shortly after launch (write-up): format: "json" sends a fixed JSON grammar, a schema gets a grammar generated from it, and invalid candidates have their logits set to negative infinity during sampling. A Hacker News commenter made the same point in 2025, describing GBNF as “tightly integrated into the token sampling infrastructure” and the base for Ollama’s JSON schema feature (thread).

Three consequences follow, and each one explains a class of bug reports:

1. The model can’t see the schema. llama.cpp’s docs say it plainly: “The JSON schema is only used to constrain the model output and is not injected into the prompt.” Clayton found the same in Ollama. A model that doesn’t know the structure gets pushed toward low-probability tokens, which shows up as odd values or runs of whitespace. Describe the structure in the prompt.
2. Shape is guaranteed, content is not. In an r/LocalLLM test of eight models at 4B or under, the poster reported that even a 1B model never broke the schema with grammar sampling; a reply pointed out that this shows the sampler works, not that the 1B model extracts the right values (thread).
3. Not all of JSON Schema compiles. llama.cpp’s converter lists gaps: prefixItems is broken, patternProperties, uniqueItems, not and if/then/else are unsupported, pattern must start with ^ and end with $, and min/max bounds apply to integers only. It also warns that “unsupported features are skipped silently.” Ollama doesn’t publish its own list, so treat llama.cpp’s as the closest guide and keep schemas plain: objects, arrays, strings, numbers, enums, required fields.

The same machinery is available directly in llama.cpp if you outgrow Ollama: llama-server accepts json_schema on completion endpoints, response_format on /chat/completions, and -j / --json-schema on the command line, as covered in the llama.cpp server guide.

Why is my Ollama structured output not working?

Most failures come from four places: the wrong format value, thinking models, a generation that stops before the JSON closes, or a client library between you and Ollama. The table collects the causes reported on GitHub and Hacker News.

The issue threads also show how version-specific this is. In #14645, commenters reported the Qwen 3.5 bug still present across releases 0.18.2 to 0.21.0, and one user stuck on 0.21.1 for driver reasons asked why the fix wasn’t backported, with no reply. If you pin Ollama, pin a version you have tested with your model and your schema.

One more interaction to know about: a Hacker News commenter in September 2026 wrote that Ollama’s structured outputs are “still incompatible with tool calling and with reasoning” (comment). The gpt-oss and Qwen fixes above narrow the reasoning half, but if you need a model to call tools and return a schema in the same turn, test it before you build on it. Tool calling itself is covered in Ollama tool calling.

Which models follow schemas best in Ollama?

Because the grammar enforces shape, the practical difference between models is the quality of the values, not whether the JSON parses. Community reports are mixed, and none of them is a controlled benchmark:

• Small models can produce valid JSON. In the r/LocalLLM test above, eight models of 4B or less on an 8 GB machine kept to the schema through LM Studio’s llama.cpp backend. One reply reported Qwen3.6-4B at Q6 producing JSON reliably. A Hacker News user wrote in early 2025 that passing a Zod schema to Ollama “works even with very small models” (comment).
• Small models can still extract badly. Another Hacker News developer, building an ontology extraction tool on Ollama, reported that models under 14B “struggle with structured output” in their experience (comment). That is a content-quality complaint, which no grammar can fix.
• Reasoning models need a recent Ollama. gpt-oss and the Qwen 3.5 series carry the bugs in the table above; on current releases both are reported working.
• Some models emit text the grammar has to fight. The r/LocalLLM poster noted a Qwen 3.5 4B variant that writes a reasoning block before the JSON, which had to be stripped before parsing.

The selection rule that follows: pick the model on extraction quality for your own documents, run a few dozen real inputs through it with the schema in place, and measure the share that passes Pydantic or Zod validation and the share with correct values. For coding pipelines that chain these calls, the local LLM coding agent guide covers model choice.

What do developers report in practice?

Developers who rely on it treat Ollama structured output as a shape guarantee plus a validation step, not as a correctness guarantee. The pattern across the sources:

• The 2024 launch thread on Hacker News drew 265 points and 70 comments (thread), and the recurring reply since has been to stop prompting for JSON and use the schema field. One r/LocalLLM commenter argued that people should stop trying to coax valid JSON out of models through prompting at all.
• Reports of bugs cluster around new model families with thinking templates, and fixes land as Ollama releases, sometimes months later (#11691 was opened in August 2025 and closed in September 2026).
• People who hit a wall either move to /api/generate, strip reasoning before parsing, or switch to llama.cpp directly, as the #14645 reporter did.

FAQ