Whattemperatureandsamplingsettingsshouldyouuseforcoding?

Most pages on LLM temperature for coding still say “use 0 to 0.3”. That was reasonable advice for 2023 chat models. It is wrong for most of the open coding models people run today. The model makers now publish their own sampling settings, and for thinking models those settings sit between 0.6 and 1.0. Start from the model card, then change one parameter at a time.

What does temperature actually do?

Temperature rescales the model’s next-token probabilities before one is picked: below 1.0 it sharpens the distribution toward the top token, above 1.0 it flattens it toward the tail.

At every step the model produces a score (logit) for each token in its vocabulary. The sampler divides those scores by the temperature and converts them into probabilities. At temperature 0 most runtimes skip sampling and take the top token every time, which is called greedy decoding. At 1.0 you sample from the distribution the model actually learned.

Two things follow for code:

• Low temperature is not “more correct”. It only makes the model pick its favorite token more often. If the favorite is wrong, you get the same wrong answer every run.
• The tail is where syntax breaks. A token at 1% probability is usually a typo, a hallucinated method or the wrong bracket. That is why the useful lever for code is often truncation (top_k, top_p, min_p), not temperature alone.

Research backs the idea that a single fixed temperature is a blunt tool for code. The AAAI 2024 paper Hot or Cold? found that code tokens split into a few hard-to-predict tokens, mostly at the start of a block, and many tokens the model is confident about. Their adaptive method raises temperature for the hard tokens and lowers it for the confident ones, and it beat fixed-temperature decoding on their benchmarks.

What LLM temperature for coding do model makers recommend?

The recommended settings range from 0.15 to 1.0 depending on the model, and the newer thinking models cluster at the high end.

These values come from each model’s card or official repository, fetched on 2026-11-04. Check the card for the exact version you run, because Qwen changed its recommendations between 3.5, 3.6 and 3.8.

A few details matter more than the numbers:

1. Qwen3.8 has no separate coding preset. Its card says the coding benchmarks were run at temperature 1.0 and top_p 0.95, the same as general thinking mode. Qwen3.6 did have a lower “precise coding” preset at 0.6. If you upgraded and kept the old settings, you are no longer on the recommended values. Our Qwen Coder local guide lists the settings for each Coder generation.
2. Google says one setting for everything. The Gemma 4 card calls temperature 1.0, top_p 0.95, top_k 64 a “standardized sampling configuration across all use cases”.
3. Devstral is the outlier. Mistral does not label it “recommended”, but its generation_config.json and every example on the card use 0.15. Runtimes that read that file inherit it.
4. Benchmarks are tied to the settings. A score published at temperature 1.0 tells you little about the model at 0.2.

What are top_p, top_k and min_p, and do they matter for code?

They decide which tokens are allowed to be sampled at all, and for code they matter as much as temperature because they cut the low-probability tail where broken syntax lives.

• top_k keeps the k most likely tokens. Qwen uses 20, Gemma 64. It is a fixed count, so it keeps junk when the model is confident and cuts good options when it is not.
• top_p (nucleus sampling) keeps the smallest set of tokens whose probabilities add up to p. At 0.95 it drops the bottom 5% of probability mass. OpenAI’s API reference recommends changing either temperature or top_p, not both.
• min_p keeps tokens whose probability is at least min_p times the top token’s probability. With min_p 0.1 and a top token at 50%, anything under 5% is cut. When the model is confident, the cutoff rises. When it is unsure, the cutoff falls and more options survive.

min_p was presented as an oral at ICLR 2025 (Nguyen et al.), which reported better quality and diversity than top_p, mostly at high temperatures. A June 2025 reanalysis (Schaeffer et al.) argued the original evidence does not support that claim once the number of hyperparameters is controlled for. The practical reading: min_p is a reasonable tail filter, not a quality upgrade. Model makers mostly set it to 0 and rely on top_p and top_k.

Sampler order also changes the result. In llama.cpp’s default chain (penalties;dry;top_n_sigma;top_k;typ_p;top_p;min_p;xtc;temperature), truncation runs first and temperature runs last, so a high temperature can only reshuffle tokens that already survived the filters.

Disabling a filter is not free. In llama.cpp’s gpt-oss guide discussion, one user reported generation speed dropping from about 57 to 35 tokens per second after setting min_p to 0.0 (llama.cpp discussion #15396). The maintainer’s guide warns that turning off top_k adds CPU overhead for the same reason: the sampler has to consider the whole vocabulary.

Should you use repetition penalties for code?

No, not by default. Code repeats tokens legitimately, and penalties push the model away from the variable names, brackets and imports it needs to reuse.

The four penalties work differently:

The gpt-oss guide in the llama.cpp repository says it plainly: “Do not use repetition penalties!” It also warns that some clients turn them on by default. Qwen’s current cards keep repetition_penalty at 1.0 and use presence_penalty only as a knob for endless repetition, with the warning that higher values can cause language mixing and a small performance drop. The one exception in our table is the older Qwen3-Coder-30B-A3B-Instruct card, which suggests a light 1.05.

How do reasoning models change the advice?

Reasoning models are trained and evaluated with sampling on, and greedy decoding makes them loop.

The Qwen3 cards are explicit. The Qwen3-8B card says: “DO NOT use greedy decoding, as it can lead to performance degradation and endless repetitions.” The DeepSeek-R1 card recommends 0.5 to 0.7 (0.6 preferred) “to prevent endless repetitions or incoherent outputs”. The mechanism is simple: a long chain of thought gives the model thousands of chances to fall into a loop, and at temperature 0 it can never sample its way out.

Hosted reasoning models often take the choice away. Developers calling GPT-5 through the OpenAI API reported a 400 error, “Unsupported parameter: ‘temperature’ is not supported with this model”, with only the default value of 1 accepted (OpenAI developer forum, Aug 2025). With local models you keep the dial, so the burden of using it correctly is yours. For gpt-oss specifically, our gpt-oss Ollama guide covers the high-effort looping reports and the fix.

What do developers report in practice?

Most “this model is dumb” reports on local coding models trace back to sampling settings, broken quants or both, and the fix is usually to return to the card’s values.

• Stacked penalties broke a coding model. An r/LocalLLaMA user running Qwen3-Coder-Next in llama.cpp with presence penalty 1.1, frequency penalty 0.5 and DRY 0.5 got death loops. Commenters called it “triple-penalizing repetition” in code, and the poster later confirmed the penalty flags were impairing the model; a quant swap also helped (thread).
• Card settings fixed looping. A Hacker News commenter running models offline on a flight said Qwen3.5 looped until they set the sampling parameters from the model card (HN, Apr 2026). Another described Qwen 3.5’s early weeks as “plagued by some premature quant releases and unclear/incomplete guidelines for the sampling parameters” (HN, Mar 2026).
• Temperature 0 trades quality for loops. In an r/LocalLLaMA thread asking whether recommended parameters are too high, one commenter said temperature matters more than all other parameters combined, and another reported that some models write better code at 0 but loop on many prompts (thread, Apr 2026).
• Agents prefer a wrong answer to a stuck one. A developer using GPT-5.2 at high temperature for coding said failed compiles are expected and fed back to the model, and they would rather get broken code than “infinite tool calling loops” (HN, Feb 2026).

These are community reports, not controlled tests. They agree on one point: the defaults of your runtime are rarely the defaults of your model.

Where do you set these in Ollama, llama.cpp and LM Studio?

Each runtime has its own defaults and its own place to override them, and none of them reads every value from the model card automatically.

Ollama. Bake the card’s values into a model with a Modelfile. Our Ollama Modelfile guide covers the full syntax.

cat > Modelfile <<'EOF'
FROM qwen3.8:27b
PARAMETER temperature 1.0
PARAMETER top_p 0.95
PARAMETER top_k 20
PARAMETER min_p 0
PARAMETER repeat_penalty 1.0
EOF
ollama create qwen3.8-code -f Modelfile

Library models often ship with parameters already set. Run ollama show --parameters <model> before you assume Ollama’s global defaults apply.

llama.cpp. Pass the values as flags when you start the server. The llama.cpp server guide covers the rest of the launch options.

llama-server -m gpt-oss-20b-mxfp4.gguf --jinja \
  --temp 1.0 --top-p 1.0 --top-k 0 --min-p 0.01 --repeat-penalty 1.0

The --top-k 0 and --min-p 0.01 pair follows the gpt-oss guide’s caveat: OpenAI’s values turn the filters off, while a tiny min_p keeps a tail cut for speed. The /v1/chat/completions endpoint also accepts top_k, min_p and repeat_penalty in the request body, which overrides the flags per request.

LM Studio. Set sampling values in the app for the loaded model, and check them after every model switch. Over the API, the OpenAI-compatible endpoint at http://localhost:1234/v1 accepts temperature, top_p, top_k, presence_penalty, frequency_penalty and repeat_penalty. The native POST /api/v1/chat endpoint also takes min_p.

One more trap: your editor or agent may send its own temperature and override everything above. Check what your client sends before you tune the server. System prompts are a separate lever, covered in our guide to the system prompt for a coding agent.

FAQ