Whitepaper 01 · 2PL IRT

Predictive Validity using Item Response Theory

Classical Test Theory treats every question as equal. Lumi does not. We model each item's difficulty and discrimination to recover a student's true latent ability: the signal that actually forecasts exam-day performance.

1. Why a raw score lies to you

Under Classical Test Theory, a student who answers 30 of 50 questions correctly earns a 60%, regardless of which 30. But a student who solves the 30 hardest questions has demonstrably more ability than one who solves the 30 easiest. A raw percentage throws that information away, and with it, any hope of an honest forecast.

Item Response Theory reframes the problem. Instead of scoring the test, we estimate a latent trait, ability, denoted $\theta$ , that explains the pattern of right and wrong answers across items of known difficulty.

2. The two-parameter logistic model

Lumi calibrates a 2PL model in which the probability that a student with ability $\theta$ answers item $i$ correctly is:

P(\text{correct}\mid\theta) = \frac{1}{1 + e^{-a_i(\theta - b_i)}}

(1)

$b_i,\ \text{difficulty}$: The ability level at which a student has a 50% chance of solving the item. Higher b = harder question.
$a_i,\ \text{discrimination}$: How sharply the item separates strong from weak students. High-a items are the ones that tell us the most about θ.
$\theta,\ \text{ability}$: The student's latent ability, fitted by maximum likelihood across every answer. Each new answer sharpens it.

Figure 1 · Item characteristic curvesIllustrative items · real model

Easy · a 0.8 · b -1.5

P 0.83

I 0.09

Medium · a 1.2 · b 0

P 0.65

I 0.33

Hard · a 2 · b +1.2

P 0.20

I 0.63

Ability θ+0.5

Figure 1.Equation 1 for three illustrative items. Each crosses 0.5 at its difficulty

b

; the steeper the curve, the higher its discrimination

a

I

is the item's Fisher information

a^2P(1-P)

at the marked

\theta

: the hard item says almost nothing about a weak student and a great deal about one near its

b

3. Calibrated on real attempts

We estimate per-item $(a, b)$ parameters from students' real attempts, refining them incrementally as answers arrive. Each student's $\theta$ is then updated continuously as they practice, and every answer is evidence that sharpens the estimate.

\theta \leftarrow \theta + \frac{\sum_i a_i\,(u_i - P_i)}{\sum_i a_i^2\,P_i\,(1 - P_i)}

(2)

\mathrm{SE}(\theta) = \frac{1}{\sqrt{\sum_i a_i^2\,P_i\,(1-P_i)}}

(3)

The maximum-likelihood step Lumi's model repeats from $\theta = 0$ , with $u_i$ = 1 for a right answer and 0 for a wrong one, and the standard error drawn as the bracket in Figure 2.

The result feeds Lumi's score predictor: a projected NEET or JEE mark built from measured ability rather than a raw percentage, so the number reflects how hard the questions you got right really were.

Figure 2 · Same score, different studentsIllustrative items · real model

Two answer sheets · scored two waysTap to flipTap a cell to flip it

Percent correct · both students

6/10·60%

Identical by percent. Not the same student.

Student AEasiest → hardest

Student BEasiest → hardest

Ability · 2PL estimate, θ ± 1 SE

A +0.34 ± 0.65·B +1.01 ± 0.61·gap 0.67

Next question Lumi would pick

Candidate items and their selector score for each student
Item	Difficulty	Fit A	Fit B
Q1	-1.0	0.66	0.47
Q2	-0.3	0.94	0.75
Q3← A	0.3	0.95	0.84
Q4← B	0.9	0.75	0.95
Q5	1.6	0.38	0.70

Fit = 0.6 × information + 0.4 × closeness to P = 0.65 · curves P = σ(a(θ − b))a and b illustrative · θ solved live

Figure 2.Two answer sheets with the same 6 of 10, scored by Equations 2 and 3. Tap any cell to flip an answer and both estimates re-solve. Below them, the item each student would be shown next.

Next: Memory decay All research

Lumi for institutes