CanyouusealocalLLMforcodereviewonyourowninfrastructure?

Why self-host code review?

Teams self-host code review when they cannot send diffs to a third party. A pull request diff is source code, often with surrounding context, and a hosted review bot sees all of it. If your contracts, regulator or threat model rule that out, the options are a vendor with zero data retention terms or a model you run yourself.

The other drivers are mundane:

• Cost at volume. A bot that reads every push on a busy monorepo burns tokens continuously. Local inference trades that for hardware you already own.
• Stability. A pinned open-weight model doesn’t change behavior between Tuesday and Wednesday.
• No refusals on security code. Reviewing auth, crypto or exploit-adjacent code is where hosted assistants sometimes decline. See why LLMs over-refuse for the mechanics.

If you want the broader infrastructure picture, including GPUs, serving and access control, self-hosting an LLM for a dev team covers it. This page stays on review.

Which open-source review tools support local models?

Three open-source reviewers document support for self-hosted models: PR-Agent, Mira and Kodus. Each reaches a local model differently, so check the endpoint story before you commit.

Two notes on that table. First, “supports Ollama” and “works well with Ollama” are different claims. PR-Agent’s own docs say local open-source models are suitable for experimentation and learning, mainly for its ask command, and recommend commercial models for production code analysis. Second, AGPLv3 matters if you modify and expose the service to other users, so have legal read it before you fork Kodus.

SaaS reviewers like CodeRabbit or Greptile exist, but they send your diff to their servers, so they sit outside this page’s scope.

Which models work best for review?

Pick a coder model with a long context window and enough active parameters to follow cross-file logic, then test it on your own pull requests. No public benchmark ranks local models on review quality for your stack.

Practitioners are the best guide here. Karsten Nohl of Security Research Labs reviewed two web apps with Qwen3.6-35B-A3B on a 32 GB MacBook M4 and got more raw findings than a Claude Opus 4.6 baseline, though he is explicit that finding counts are not validated bugs. A Houtini write-up went the other way and dropped a 35B-A3B model from reviews because it invented line references.

Qwen3-Coder-30B-A3B-Instruct is a reasonable starting point. Its model card lists 30.5B total parameters with 3.3B activated per token, a native context of 262,144 tokens, and an Apache-2.0 license. A mixture-of-experts model like this is cheap to run per token, which matters when a bot reviews every push. For hardware sizing, see local LLM hardware requirements and the Qwen coder guide.

A few selection rules that hold up across the sources below:

1. Context first. The diff, the files it touches and the prompt all have to fit. Size the window for your largest typical PR, not your average one.
2. Prefer a model that doesn’t invent line numbers. One practitioner excluded a 35B-A3B model from reviews because it invented line references (see below).
3. Run two different families if you can. Different models miss different things. SRLabs found the union of findings across models was far larger than any single model’s.
4. Pin the version and the quant. A silent model swap changes your false-positive rate.

What does research say about LLM code review quality?

Research says LLMs are useful reviewing assistants and unreliable autonomous bug detectors. Two 2026 studies make the limits concrete.

• Waterloo thesis, open-weight models. Ayinde Yakubu evaluated gpt-oss-120b, gemma-4-31B-it and Qwen3.6-35B-A3B on the ApacheJIT dataset for Apache Kafka: 524 bug-inducing commits and 530 clean ones, zero-shot, three runs each. Per the presentation abstract, the best recall was 0.5163, accuracy never exceeded a trivial always-NOBUG baseline, balanced accuracy stayed below 0.5 and MCC was negative in the reported runs. Only about 16 to 23% of true-positive explanations in the first run matched the documented defect. The author suggests evaluating assistant-style use instead. This is a thesis presentation page, not a peer-reviewed paper.
• Overcorrection. Jin and Chen (arXiv 2603.00539) report that LLMs often flag correct implementations as non-compliant, and that prompts demanding explanations and proposed fixes increase those wrong judgments. Their abstract doesn’t name the models in the text I could read.

The practical reading: a bot that comments on every PR will be wrong some of the time, and asking it to be more thorough can make it more wrong. That is a reason to constrain the prompt, not a reason to skip review automation.

What do developers report in practice?

Practitioners report real catches mixed with confident errors, and they say prompt discipline matters more than model size. These are community and practitioner reports, not our tests, and each is a single setup.

• Security review on a laptop. Karsten Nohl of Security Research Labs ran Qwen3.6-35B-A3B and Gemma4-26b through LM Studio on a MacBook M4 with 32 GB, reviewing two web apps of about 150 and 85 files. Qwen produced 49 raw findings on the first app against 43 for a Claude Opus 4.6 baseline, and 34 versus 26 on the second, in under 90 minutes each. He is explicit that finding counts are not validated bugs: Qwen rated an authorization flaw Low that another model rated Critical, and a “Source-local” design still sends metadata such as file trees and routes to the cloud.
• A real bug and a confident error. A Houtini write-up on a 4-bit Qwen3.8-27B model on a 48 GB card at 32k context reports 35.8 tokens/second. On a 142-line file with two known bugs the model found neither across three runs. A prompt labelled “exhaustive expert” produced about 12 false findings on an earlier model, while the author’s own 183-word prompt produced none. The 35B-A3B model was excluded because it invented line references. The author labels several of these results as unproven.
• Context starvation. The editorial guide from LLM Configurator calls narrow diffs the top cause of false positives and recommends --unified=15 plus an explicit “do not report” list. It cites no data for that claim, so treat it as advice, not a finding.

A Show HN thread for Mira is a useful caution about vendor numbers: its benchmark was run on a hosted Claude model, not a local one, so it says little about local quality.

What does a CI pipeline with a self-hosted endpoint look like?

The pattern is a CI job that sends the diff to an inference server you control and posts the result back to the pull request. Three parts matter: where the model runs, how the job reaches it, and how much context it gets.

1. Serve the model on a GPU box behind your firewall with Ollama, vLLM or llama.cpp. See Ollama vs llama.cpp vs vLLM for the choice.
2. Make it reachable from the runner. A GitHub-hosted runner cannot see a private endpoint, so use a self-hosted runner inside your network or an authenticated gateway. This is a design consequence, not a documented PR-Agent requirement.
3. Point the reviewer at the endpoint. PR-Agent’s docs show this for Ollama:

[config]
model = "ollama/qwen2.5-coder:32b"
custom_model_max_tokens = 128000

[ollama]
api_base = "http://localhost:11434"

4. Fix the context window. PR-Agent’s docs warn that Ollama defaults to a 2,048-token context, too small for its prompt plus a diff, and tell you to raise it and keep custom_model_max_tokens in sync:

OLLAMA_CONTEXT_LENGTH=8192 ollama serve

A mismatch between the two values “may result in unexpected model output” per the same page. Size the number for your real PRs, and remember the KV cache grows with it.

5. Trigger it from CI. PR-Agent documents a GitHub Action (the-pr-agent/pr-agent@main) as its recommended route. Mira and Kodus install as apps or Docker services instead.

How do you keep humans in the loop?

Keep a human approver on every merge and treat the bot’s output as a checklist of things to look at. The studies above don’t support autonomous gating.

• Comment, don’t block. Post findings as review comments and don’t set a required status until you have measured the false-positive rate on your own repo.
• Constrain the prompt. Tell the model what to skip (formatting, naming, anything a linter catches) and ask for a fixed output format with severity. Short prompts beat “exhaustive expert” prompts in the Houtini report.
• Run static tools alongside. SRLabs paired the local model with Semgrep, npm audit and Brakeman. Deterministic tools catch what the model’s confident prose misses.
• Log what the bot said and what reviewers did with it. The accept rate of its comments is your only honest quality metric.
• Keep the AI in the loop on your terms. For the trade-off between this and a hosted model, see local LLM vs Claude for coding.

FAQ