arXiv:2605.15015 Small Private LM: kompetitivni rezultati u educational assessment design uz human-in-the-loop preporuke

Chris Davis Jaldi, Anmol Saini, Shan Zhang, Noah Schroeder, Cogan Shimizu i Eleni Ilkou objavili su 14. svibnja 2026. na arXivu paper koji adresira jednu od kritičnih praznina trenutnog AI-in-education diskursa — kako koristiti AI za assessment design uz privacy guarantees koje educational sektor zahtijeva.

Što je educational assessment design problem?

Generativni AI je demonstrirao impressive sposobnost generiranja pedagogically aligned questions — quiz pitanja, problem sets, essay prompts koji ciljaju specifične Bloom’s taxonomy razine. Industrija već koristi GPT-4, Claude i Gemini za ovaj task.

Problem: educational data je extremely sensitive. Student responses, learning analytics, curriculum specifics — ništa od toga ne smije završiti u cloud API logs koji se mogu koristiti za model training. Cloud-based LLM API-jevi su za škole compliance noćna mora (FERPA u SAD-u, GDPR-Art. 8 u EU-u, lokalni regulacijski okviri za maloljetnike).

Što paper konkretno demonstrira o smaller modelima?

Autori provode sistematsku komparaciju smaller modela protiv larger alternativa:

• Quality dimension — sposobnost generiranja questions usklađenih s Bloom’s taxonomy levels (remember, understand, apply, analyze, evaluate, create)
• Reproducible metrics — measurement framework koji se može independent reproducirati, ne subjective rater opinions
• Comparison to expert human judgment — model-generated questions evaluirani protiv ratings expert educatora

Findings: smaller modeli postižu kompetitivne rezultate kroz quality dimensions. Razlika nije dramatic kako se često pretpostavlja — appropriate fine-tuned 7-13B parametara model može aproximirati 70-200B model output za assessment design task.

Koje je critical limitation otkriveno?

Paper naglašava značajan caveat: “model-based evaluations also exhibit systematic inconsistencies and bias relative to expert ratings”. Praktične posljedice:

• Ako koristimo LLM-as-judge za evaluation drugih LLM outputa, akumuliramo bias kroz cijeli pipeline
• Model preferira generated questions koje liče na svoje vlastite outputs, ne nužno pedagogically optimal
• Apparent quality consensus među different modelima može biti artifact shared training data, ne real pedagogical validity

Koja je glavna preporuka?

Autori jasno preporučuju Human-in-the-Loop pristup. Konkretne implikacije:

• Mali modeli kao teammates — ne kao autonomous agents
• Expert review obavezan za final output validation
• Local deployment za privacy preservation, ali ne za circumvention of human review
• Bloom’s taxonomy alignment mora biti expert-verified, ne purely model-judged

Pristup je kompatibilan s emerging educational AI policy frameworks-ima — UNESCO, EU Digital Education Action Plan, US Department of Education AI guidelines. Svi naglašavaju AI augmentation, not replacement of educational professionals.

Što ovo znači za education tech sector?

Paper validira niche koju startup-i poput Khanmigo, Magic School AI, i open-source projekti kao OpenLLM-In-Education istražuju: small privacy-respecting modeli koji rade locally na školskoj infrastrukturi umjesto cloud API zahteva.

Pristup je commercial fit:

• Schools/universities — privacy compliance bez compromise on capability
• Edtech vendors — manji compute cost, on-premise deployment opcija
• Open-source community — fine-tuneable basis modeli (Llama, Qwen, Phi) za educational specijalizaciju

Paper se uklapa u širi 2026. trend specialized small models for sensitive domains: medical small LM-ovi (Cardio-LLM, MedFlow GraphFlow 15.5.), legal small LM-ovi, financial small LM-ovi. One-size-fits-all frontier API model ima konkurenciju iz specijaliziranih small modela koji bolje servisiraju regulated sectors s privacy demands.