I tested five decision models against 502 Korean reading comprehension questions published in previous TOPIK II exams.
JEV 1.14 had the highest accuracy at 97.0% and Tev1 4B was the fastest, at 54 milliseconds per question on average. I measured answer accuracy, request time, and commercial API cost.
Decision models select from supplied answers and return probabilities. TOPIK's four-choice reading questions provide a straightforward way to test that interface in Korean.
The dataset came from 12 publicly released papers in the official TOPIK archive, covering rounds 35, 36, 37, 41, 47, 52, 60, 64, 83, 91, 96, and 102. They contained 600 reading questions. I excluded 72 questions requiring images and 26 whose passages were omitted from the official PDFs, leaving 502 text-only questions. Listening and writing were outside the scope of this test.
Answers were verified against official answer keys.
Each model received the same Korean question, passage, and four choices in their original order. Each question was sent as a separate request, with no request for an explanation. Accuracy used all 502 questions as its denominator, including refusals.
Tev1 4B, Nimble 9B, and Clef Flash 9B ran through Ollama 0.40.13 using Q8_0 models. My computer has an RTX 5070 with 12 GB of VRAM, an Intel Core Ultra 9 285K, and 64 GB of RAM. Tev1 and Nimble ran entirely on the GPU at roughly 8K context. Clef was retested with a 2K context and all model layers on the GPU; the table reports this configuration. The hosted candidates were JEV 1.14.0 and OpenAI Decisions using gpt-6-luna.
Requests were sequential within each candidate. The initial hosted runs ran alongside the local GPU job. Clef's retest with this configuration ran separately. Timing measured the complete API round trip after a synthetic warm-up, including network overhead for hosted models. Costs were calculated from reported token usage and published rates recorded on November 1, 2026. Warm-up time and cost were recorded separately and excluded from the table. Local electricity and hardware costs were not measured.
| Model | Correct | Accuracy | Mean request time | API cost for all 502 |
|---|---|---|---|---|
| JEV 1.14 | 487/502 | 97.0% | 159 ms | $0.0131 |
| OpenAI Decisions | 477/502 | 95.0% | 282 ms | $0.0185 |
| Clef Flash 9B (2K context, GPU) | 464/502 | 92.4% | 79 ms | — |
| Nimble 9B | 458/502 | 91.2% | 95 ms | — |
| Tev1 4B | 428/502 | 85.3% | 54 ms | — |
The new Clef configuration reduced mean request time from 290 ms to 79 ms, about 3.7 times faster. The Q8_0 weights and all 502 selected answers were unchanged.
OpenAI refused two questions and I counted both against its accuracy. There were no other request failures. Including warm-ups, the estimated commercial cost was $0.01317 for JEV and $0.01850 for OpenAI, approximately 3.17 cents combined.
These results (and all benchmarks, really) have limits. Public exam questions may have appeared in model training data and this run cannot establish performance on unseen questions.