jev-korean-benchmark

What it does

How well does Jev understand Korean, including medical text? This benchmark publishes questions, results, runtime and cost.

How you can use it

Write your instructions and source passages directly in natural Korean. Translating instructions into English does not help the model read Korean text any better.

Be careful when asking the model to spot tiny differences in wording. Also, shuffling choices can change roughly one out of eight answers, so check critical results.