Qwen3 Writing Coach Benchmark: 4B vs 8B vs 14B.

Why this writing coach benchmark exists

A writing model can produce a correct sentence while giving a weak explanation. It can also identify the right rule but place the correction in the wrong field or respond too slowly for a practical local workflow. For a writing coach, those dimensions should not be collapsed into one score.

This exploratory Qwen3 benchmark was created around the same product philosophy as LinguaPilot: the user writes first, and the AI should correct, improve and explain without replacing the user’s thinking or voice.

How the benchmark was run

1. Same cases for every modelQwen3 4B, 8B and 14B received the same 20 cases, including 16 sentences with expected errors and 4 already-correct controls.
2. Clear output contractEach model had to provide a minimally corrected text, an optional improved version, detected errors and explanations in French.
3. Frozen reference comparisonModel outputs were evaluated offline against versioned expected corrections and acceptable local operations. The models did not see the hidden reference.
4. Paired case analysisThe main result was complete-case success: every expected error corrected without an incorrect replacement, false error or deterministic fact loss.
5. Cold-start local timingOllama used keep_alive=0, so the model was reloaded for each response. The timing therefore represents a cold-start workflow on the tested machine.

Main Qwen3 benchmark results

Qwen3 8B and 14B completed 19 of 20 cases, compared with 18 for 4B. At the individual correction level, 8B and 14B applied 20 of 21 expected corrections, compared with 19 for 4B.

What each Qwen3 model did well

Correction quality versus local speed

Model size changed responsiveness much more than complete-case correction. Qwen3 14B took about twice as long as 8B and about 4.5 times as long as 4B in this cold-start configuration.

Error localization and explanation quality measure different skills

The 8B model localized all 21 expected error regions in this run. The 4B and 14B models each missed one, while 14B also reported one false-positive region. This measures detection discipline, while explanation quality is evaluated as a separate dimension.

How to read the paired comparison

Qwen3 8B and 14B each completed one more case than 4B. The paired comparison shows that this observed difference came from one case, while 8B and 14B achieved the same complete-case result across the 20 tested cases.

Practical interpretation: Qwen3 8B and 14B achieved the same complete-case score in this run. Across the 20 tested cases, neither model showed a practical advantage over the other on the primary correction outcome.

Methodology and scope

• Exploratory design: the 20 paired cases provide a focused comparison of this specific writing-coach scenario rather than a universal model ranking.
• One language pair: English writing with explanations requested in French.
• One model family: the study compares Qwen3 sizes, not Qwen3 against unrelated model families.
• One machine: local timing depends on hardware, Ollama configuration and model quantization.
• Pedagogical checks: automatic checks cover selected patterns and contradictions, while deeper explanation quality remains a separate area for linguistic review.
• Evaluation reference: responses were compared with a frozen, versioned correction reference prepared specifically for this study, supporting reproducible comparison across the three models.

From controlled benchmark cases to real writing

Benchmarks describe controlled prompts. Real writing happens in Outlook, Word, browsers, Notepad and everyday messages. LinguaPilot brings a corrected version, an improved version and an explanation directly into those Windows workflows through one shortcut.

You can use Ollama locally for private or routine texts, then choose an optional cloud provider for other tasks. The goal remains the same: improve the text while keeping the user as the author.