A Method for Dialog-Based Knowledge Assessment Using a Two-Loop Human–LLM Interaction

Abstract

— This paper proposes a method for dialog-based knowledge assessment based on a two-loop interaction with a large language model (LLM), enabling the separation of subject content preparation from individual knowledge assessment. The method is organized around two interconnected iterative loops: the expert–LLM loop, intended for the generation and expert validation of examination content, and the student–LLM loop, which implements an adaptive diagnostic dialogue through the progressive refinement of the learner’s knowledge model. The paper defines principles for distributing functions among the instructor, the large language model, and the information system, whereby the LLM performs cognitively intensive tasks related to content generation and response analysis, while the final subject-specific judgment remains the responsibility of the instructor. Based on the proposed method, the architecture of an intelligent assessment system was developed, a software prototype was implemented, and its experimental validation was carried out in an educational environment. The experimental results confirm the practical feasibility of the proposed approach and demonstrate its potential to substantially reduce the effort required for developing assessment item banks and conducting individualized dialog-based knowledge assessment while preserving reproducibility, traceability, and expert reliability of the assessment results.