What an ITS Is
An intelligent tutoring system is a software system that provides individualized instruction by maintaining three explicit models:
| Model | What it represents |
|---|---|
| Domain model | What the learner is being taught: skills, concepts, relations between them, correct procedures, common errors |
| Learner model | What the learner currently knows: which skills are mastered, where the misconceptions are, what the recent evidence supports |
| Tutor model | What to do next: which problem to present, when to give a hint, when to advance, when to remediate |
The three-model architecture distinguishes ITS from simpler adaptive systems: a quiz that branches on a wrong answer is adaptive but does not have an explicit domain or learner model; a recommender that surfaces the next video does not have a tutor model. The phrase "intelligent tutoring system" should be reserved for systems with all three.
The architecture goes back to early-1970s artificial-intelligence research, most notably the work of John Self, John Anderson, and Beverly Park Woolf, with major implementations starting in the 1980s. The phrase itself was popularized by D'Mello, Graesser, and others in the 1990s and 2000s. The contemporary canonical references are Anderson, Corbett, Koedinger, and Pelletier (1995), Journal of the Learning Sciences 4(2): 167-207, on the Cognitive Tutor; VanLehn (2006), International Journal of Artificial Intelligence in Education 16(3): 227-265, on the behaviour of tutoring systems; and VanLehn (2011), Educational Psychologist 46(4): 197-221, on the effect sizes.
This page covers ITS as a method that educators and curriculum designers can use, frame, and critique. It explains the architecture, the empirical case (with the careful caveats), the canonical examples, and the relationship to TheoremPath's adaptive components.
The Architecture in Detail
The domain model
The domain model encodes what is being taught. In a Cognitive Tutor for high-school algebra, the domain model is a set of skills (apply distributive law, isolate variable, combine like terms) plus the relations among them (which skills must be mastered before others), plus a library of problems that exercise each skill in different combinations. The standard representation is a production system: a collection of condition-action rules that capture both correct steps and common errors.
Two structural choices in domain models matter for educators:
Granularity of skills. A domain model with too few skills treats different cognitive moves as the same thing; a domain model with too many skills produces unstable mastery estimates because each skill is rarely exercised. The standard practice is to map curriculum units to roughly 5-30 skills, with explicit Q-matrix mapping items to skills. The Q-matrix concept goes back to Tatsuoka (1983).
Coverage of misconceptions. Strong domain models include not just correct production rules but the common buggy rules that produce specific incorrect answers. The bug-rule library lets the tutor diagnose why the learner is wrong, not just that they are wrong. Brown and VanLehn (1980) on the arithmetic bug catalogues are the canonical reference.
The learner model
The learner model is a representation of the learner's current state with respect to the domain model. The dominant approach is BKT-style mastery tracking: a probability per skill that the learner has mastered it, updated on each item attempt. See bayesian-knowledge-tracing-for-educators for the full treatment.
Three levels of learner modelling, increasing in fidelity:
| Level | What it tracks | Example |
|---|---|---|
| Static placement | A single ability estimate, set once at the start | Pre-test driven branching |
| Skill-level mastery | One probability per skill, updated each interaction | BKT, PFA |
| Skill-level mastery + misconception state | As above, plus a separate probability for each known misconception | The Cognitive Tutor with bug rules |
Modern systems also use deep learner models (deep knowledge tracing, transformer-based models) that trade interpretability for predictive accuracy. The choice between interpretable and deep is a design decision; the empirical evidence on whether predictive accuracy translates into instructional effectiveness is mixed.
The tutor model
The tutor model decides what to do next. Three principal decisions:
- Item selection. Given the current learner state, which problem to present? The standard rules: target a skill near the boundary of mastery (high information about the learner's current state); avoid items that have just been seen (recency spacing); ensure curriculum coverage (do not skip skills the learner has not yet attempted).
- Hint policy. When the learner is stuck, what to say? The standard cascade is: nothing first; a strategic hint (what class of move is needed); a procedural hint (what specific step); a worked example (here is the answer with full reasoning). VanLehn (2011) found that "knowledgeable hints that are responsive to the learner's specific error" is one of the strongest contributors to effect size.
- Advancement and remediation. When does the system declare a skill mastered and move on? When does it back up to a prerequisite? The standard mastery threshold is 0.95 posterior probability under the BKT model, but the threshold is a design choice, not a fact about BKT.
Empirical Status
The empirical case for ITS is well-developed and important. The careful claim is also well-developed, and is what follows.
The VanLehn 2011 effect-size claim
The most-cited empirical claim is from VanLehn (2011), Educational Psychologist 46(4): 197-221:
| Tutoring condition | Approximate effect size (d, vs no tutoring) |
|---|---|
| Step-based tutoring (typical ITS) | 0.76 |
| Substep-based tutoring (more granular ITS) | 0.40 |
| One-on-one human tutoring | 0.79 |
| Worked-example study | 0.61 |
| Reading text | 0.05 |
The headline reading: ITS produces effect sizes comparable to one-on-one human tutoring, both substantially larger than the typical effect of "computer-based learning systems" without an ITS architecture. This was a substantial revision of the earlier "two-sigma problem" framing (Bloom 1984) that had treated human tutoring as an upper bound impossible for technology to match.
What the effect-size table is and is not:
- It is a meta-analytic synthesis of many studies with heterogeneous designs.
- It is comparing learning gain on knowledge measures, typically pre-post tests on the same subject matter.
- It does not compare ITS to high-quality classroom instruction; the "no tutoring" comparison condition varies across studies.
- It is averaged over subject matter, learner population, and delivery context. Specific deployments vary substantially in measured effect.
- It does not include long-term retention measures.
Steenbergen-Hu and Cooper (2014), Journal of Educational Psychology 106(2): 331-347, conducted a meta-analysis of K-12 ITS deployments and found effect sizes around 0.05-0.30 versus typical classroom instruction. The lower estimate reflects the difference between "ITS vs no help" (a stark comparison) and "ITS vs a well-resourced teacher" (a much less stark one).
The honest summary: ITS produces measurable learning gains in appropriate contexts. The size of the gain depends on what the comparison condition is, how well-implemented the ITS is, and which subject is being taught. Treating the VanLehn 0.76 figure as a universal property of ITS is an overstatement; treating the much smaller K-12 effect as evidence that ITS does not work is also an overstatement. The real effect is real and bounded.
Other empirical findings worth knowing
- ITS works best for well-defined domains. Mathematics, programming, formal-logic, anatomy: cleanly structured declarative + procedural knowledge. Open-ended writing feedback, ethical reasoning, design judgment: harder, with smaller and more variable effects.
- The hint policy matters more than the item-selection policy. Multiple comparison studies (VanLehn 2006; later reviews) find that what the system says when the learner is stuck has a larger effect on outcomes than which exact item is presented next.
- Aptitude-treatment interactions are real. Some learner populations respond strongly to ITS; others respond weakly even when the system is well-implemented. The standard predictors are domain prior knowledge (more prior helps), metacognitive skill (self-regulated learners help themselves more), and motivation.
Canonical Examples
| System | Lead developers | Domain | What is distinctive |
|---|---|---|---|
| Cognitive Tutor / MATHia | Anderson, Corbett, Koedinger; Carnegie Learning | School mathematics | Production-rule domain model with bug catalogues; BKT-style learner model; the canonical example |
| AutoTutor | Graesser et al., Memphis | Conceptual physics, computer literacy, scientific reasoning | Natural-language dialogue with the learner; multiple "agents" in the interface |
| ASSISTments | Heffernan and Heffernan, Worcester Poly | School mathematics | Open research platform; substantial public response data |
| AnimalWatch | Beal, Beck | Pre-algebra word problems | Affect-aware tutoring (responds to frustration / engagement signals) |
| Why2-Atlas | VanLehn et al. | Conceptual physics | Knowledge-construction dialogues |
| Crystal Island | Lester et al. | Microbiology | Game-based ITS |
| ITS for Programming | Multiple groups | Introductory programming | Standard test-bed; many comparison studies |
The Cognitive Tutor is the most empirically studied. ASSISTments is the most accessible for academic research because the platform is open and the response logs have been used in hundreds of educational-data-mining papers. AutoTutor is the most distinctive in interaction model.
Mechanism: Why ITS Works (When It Does)
Three structural features carry the weight:
Diagnostic feedback. A well-implemented ITS knows not just that the learner is wrong, but which buggy production rule fits their behaviour, and can give targeted feedback. This is closer to one-on-one tutoring than to grade-feedback delivery.
Fine-grained pacing. The mastery loop advances the learner when the evidence supports advancement and holds them when it does not. Compared to fixed classroom pacing, this prevents both the "lost in the curriculum" experience (the learner moves on without mastering the prerequisite) and the "bored by review" experience (the learner sits through material they already know).
Substantial practice volume. Even modest ITS deployments typically generate more practice problems per learner than classroom instruction. Practice volume by itself produces learning gains; the ITS architecture amplifies them.
What the architecture does not do, by itself: motivate disengaged learners, replace teacher relationship, teach metacognition, build collaborative skill, or provide the authentic-context learning that some subjects require.
Boundary Conditions: Where ITS Falls Short
Five places to be careful.
Domain limits. ITS works best where the domain has well-defined skills, identifiable correct answers, and a reasonable number of common error patterns. Open-ended domains (essay writing, design judgment, ethical reasoning) are harder. The standard remedy is hybrid systems where ITS handles the well-defined component (grammar, structure, factual content) and human instruction handles the open-ended component.
Cold start. The learner model is uninformative until the learner has produced a meaningful number of responses. Pre-tests partially address this; carefully chosen first problems do too. Nothing fully addresses it.
Affect and engagement. Learners disengage from ITS at higher rates than from teacher-led classrooms. Affect-aware tutoring (D'Mello and Graesser 2014) is an active research area; the deployed solutions are still incomplete. This is the most important practical limit on ITS at scale.
Calibration depends on data. A new domain model needs extensive piloting to calibrate item difficulties, identify common bugs, and tune the mastery thresholds. Building a new ITS from scratch is a substantial multi-year undertaking; this is why the field has only a few dozen serious examples.
Replacement narrative. ITS does not replace teachers or classroom instruction. The successful deployments are supplements: ITS provides individualized practice while classroom time is used for instruction, discussion, and projects. Deployments that try to use ITS as a complete replacement consistently produce worse outcomes than blended deployments.
How TheoremPath's Adaptive Components Relate to the ITS Tradition
TheoremPath has some of the ITS architecture and not others. The honest mapping:
| ITS component | TheoremPath equivalent |
|---|---|
| Domain model | The prerequisite DAG (content/topics/*.mdx frontmatter), the Q-matrix mapping items to skills (data/content/q-matrix.json), the skill graph (data/content/skills.json) |
| Learner model | TopicAssessment + AssessmentAttempt (BKT-flavoured); the LearningEvent append-only log; the difficulty-aware mastery scoring in src/lib/mastery/scoring.ts |
| Tutor model | Partial. Topic recommendations exist via the RecommendationEvent log; the RecommendationEvent model captures policy decisions. The full "next-best item under current state" tutor loop is not yet productized; the substrate to log the decisions is in place |
| Hint / explanation cascade | Not implemented as a tutor-loop component. Each MDX page is a complete explanation; there is no item-level interactive hint policy |
| Affect awareness | Not implemented |
| Bug catalogue / misconception model | Partial. Each multiple-choice question has explanations for wrong answers in its YAML; this is the foundation of a misconception model but is not consumed by an automated tutor loop |
So TheoremPath has the substrate of an ITS without the tutor
loop. The substrate framing is documented in
docs/ADAPTIVE_LEARNING_KERNEL.md, which is explicit about this:
the system is internal-only, the policy problem is treated as
contextual bandit / rules-plus-evaluation rather than full RL,
and durable action / impression logging for policy work is
identified as a future need.
This is a deliberate design choice. The ITS tradition has shown that domain modelling, learner modelling, and tutor-loop policy each take substantial work to do well; building all three at once typically produces a bad version of each. TheoremPath has prioritized domain modelling and learner modelling to get to useful surfaces (mastery dashboards, review queues, prerequisite- aware navigation) before the tutor-loop layer.
Common Misapplications
Calling something an ITS that lacks the architecture. A quiz with branching is not an ITS. A chatbot that answers homework questions is not an ITS. A recommender that surfaces the next video is not an ITS. The phrase has a specific meaning and educators should hold to it.
Assuming the VanLehn 0.76 effect transfers. The effect depends on context. Citing the figure to motivate a specific ITS deployment is reasonable; citing it as an unconditional property of "AI tutors" is overstatement.
Treating "AI tutor" as synonymous with ITS. Most current "AI tutor" products are LLM wrappers: a chat interface with prompt engineering and possibly retrieval-augmented generation. Some are useful; few have the domain model, learner model, and tutor model an ITS requires. The terminology is becoming muddled and educators should resist letting "AI tutor" do the work of "ITS" in serious discussion.
Skipping the calibration work. Deploying a new ITS without substantial pilot calibration produces unstable mastery estimates, miscalibrated item difficulties, and a brittle hint policy. The Cognitive Tutor's measured effect rests on years of calibration; new systems do not get that for free.
Treating it as a teacher replacement. Successful deployments supplement teachers, not replace them. The "all-ITS, no classroom" model has been tried and produces worse outcomes than blended deployments.
Related Methods
| Method | What it is | Relation to ITS |
|---|---|---|
| ITS | Three-model adaptive instruction system | The full architecture |
| Adaptive testing (CAT) | Item-selection driven by IRT | The item-selection component of the tutor model, in a testing rather than learning context |
| Bayesian Knowledge Tracing | Skill-mastery probability tracking | The standard learner-model component |
| Item Response Theory | Per-item calibration | The standard item-calibration component |
| Spaced repetition (FSRS) | Review scheduling | A separate component, often paired with ITS for review queues |
| Recommender systems | Next-item prediction from collaborative-filtering | A simpler alternative to the tutor model; lacks the domain / learner model |
| LLM-based "AI tutors" | Conversational systems backed by large language models | Often lack the explicit three-model architecture; useful but not equivalent |
The pages on bayesian-knowledge-tracing-for-educators, item-response-theory-for-educators, and fsrs-spaced-repetition-for-educators cover the canonical components. The the-theorempath-pedagogy-thesis page covers how the components combine in the TheoremPath architecture.
What This Page Does Not Claim
This page does not claim ITS is a solved problem. The three-model architecture is well-understood; building a production-quality instance for a new domain remains substantial work.
This page does not claim LLM-based tutoring is or is not an ITS. Some LLM products are starting to integrate explicit domain and learner models; others remain pure conversational interfaces. The phrase "ITS" should be applied based on architecture, not branding.
This page does not claim TheoremPath is an ITS. TheoremPath has
the domain and learner-model substrate of an ITS, with the
tutor-loop layer deliberately deferred. The site is honest
about this in docs/ADAPTIVE_LEARNING_KERNEL.md.
FAQ
What's the difference between an ITS and adaptive learning?
"Adaptive learning" is a marketing umbrella that includes ITS, spaced repetition, recommendation engines, and various quiz-with-branching systems. ITS is a specific architecture within that umbrella, defined by the three-model structure.
Are LLM-based AI tutors ITSs?
In principle they could be (an LLM wrapper plus an explicit domain model plus an explicit learner model plus a tutor-loop policy is an ITS). In practice most current products are LLM wrappers without the architectural pieces, and labelling them "ITS" is loose usage. The right test is to ask: does the system maintain an explicit learner model that survives the conversation, and does it have an explicit policy for what to do next based on that model?
What's the gold-standard ITS to study?
The Cognitive Tutor / MATHia, by Anderson, Corbett, and Koedinger. Anderson et al. (1995) is the canonical paper; Ritter, Anderson, Koedinger, and Corbett (2007) covers the Cognitive Tutor as a deployed product; the Carnegie Learning materials describe the production system.
How big is the ITS effect really?
In the conditions VanLehn (2011) reviewed, around 0.76 standard deviations versus a no-tutoring comparison; in K-12 large-scale deployments versus typical classroom instruction (Steenbergen-Hu and Cooper 2014), around 0.05-0.30. Both numbers are real; they answer different questions.
Can ITS work without BKT?
Yes. The three-model architecture is more general than any specific learner-modelling technique. Performance Factor Analysis, Deep Knowledge Tracing, and other learner models are all ITS-compatible. BKT is the most-deployed because of its interpretability and tractability, not because it is the only option.
How does this connect to TheoremPath?
TheoremPath has the domain and learner-model substrate of an ITS: the prerequisite DAG, the Q-matrix, the skills graph, and the BKT-flavoured mastery layer. The tutor-loop layer is deliberately deferred; the substrate to support it (the LearningEvent and RecommendationEvent logs) is in place. The the-theorempath-pedagogy-thesis page covers the design choice in detail.
Internal links
- PedagogyPath: bayesian-knowledge-tracing-for-educators for the canonical learner-model component; item-response-theory-for-educators for item calibration; fsrs-spaced-repetition-for-educators for the review-scheduling layer that pairs with ITS; the-theorempath-pedagogy-thesis for how the four pillars combine in TheoremPath's architecture.
- TheoremPath: the existing reinforcement-learning-from-human-feedback-deep-dive page for the contemporary policy-learning context that contextual-bandit ITS sits inside.
- PhilosophyPath: plato-as-teacher for the older one-on-one tutoring tradition that VanLehn's effect-size comparison invokes.
Sources and further reading
Foundational:
- Anderson, J. R., Corbett, A. T., Koedinger, K. R., and Pelletier, R. "Cognitive Tutors: Lessons Learned." Journal of the Learning Sciences 4(2) (1995): 167-207. The canonical paper on the Cognitive Tutor.
- Brown, J. S., and VanLehn, K. "Repair Theory: A Generative Theory of Bugs in Procedural Skills." Cognitive Science 4(4) (1980): 379-426. The bug-rule analysis foundation.
- Tatsuoka, K. K. "Rule Space: An Approach for Dealing with Misconceptions Based on Item Response Theory." Journal of Educational Measurement 20(4) (1983): 345-354. The Q-matrix concept.
Empirical:
- Bloom, B. S. "The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring." Educational Researcher 13(6) (1984): 4-16. The original "two-sigma" framing.
- VanLehn, K. "The Behavior of Tutoring Systems." International Journal of Artificial Intelligence in Education 16(3) (2006): 227-265.
- VanLehn, K. "The Relative Effectiveness of Human Tutoring, Intelligent Tutoring Systems, and Other Tutoring Systems." Educational Psychologist 46(4) (2011): 197-221. The canonical effect-size review.
- Steenbergen-Hu, S., and Cooper, H. "A Meta-Analysis of the Effectiveness of Intelligent Tutoring Systems on K-12 Students' Mathematical Learning." Journal of Educational Psychology 106(2) (2014): 331-347.
Affect-aware tutoring:
- D'Mello, S., and Graesser, A. "Confusion and Its Dynamics during Device Comprehension with Breakdown Scenarios." Acta Psychologica 151 (2014): 106-116. One of the affect-aware tutoring entries.
Specific systems:
- Ritter, S., Anderson, J. R., Koedinger, K. R., and Corbett, A. "Cognitive Tutor: Applied Research in Mathematics Education." Psychonomic Bulletin & Review 14(2) (2007): 249-255. The Cognitive Tutor as deployed product.
- Graesser, A. C., Conley, M. W., and Olney, A. "Intelligent Tutoring Systems." In K. R. Harris, S. Graham, and T. Urdan (Eds.), APA Educational Psychology Handbook, vol. 3. American Psychological Association, 2011.
- Heffernan, N. T., and Heffernan, C. L. "The ASSISTments Ecosystem: Building a Platform that Brings Scientists and Teachers Together for Minimally Invasive Research on Human Learning and Teaching." International Journal of Artificial Intelligence in Education 24(4) (2014): 470-497.
This page is part of PedagogyPath, sister site to TheoremPath in the path-network family. It is one of the four ITS / adaptive- component pages that explain the empirical and architectural basis for TheoremPath's adaptive infrastructure; the canonical statement of how the components fit together is at the-theorempath-pedagogy-thesis.