G-Theory D-Study Calculator
Enter variance components from a balanced G-study to project relative and absolute reliability for a planned items-by-raters design. The calculator also searches for the smallest item and rater counts that can reach your target.
Absolute Phi coefficient82.05%
- Relative error variance
- 7
- Absolute error variance
- 8.75
- Relative G coefficient
- 85.11%
- Absolute Phi coefficient
- 82.05%
- Relative SEM
- 2.65
- Absolute SEM
- 2.96
- Minimum items at current rater count
- No finite fixed-facet solution
- Minimum raters at current item count
- 7
- Lowest-workload target design
- 13 items x 5 raters (65 ratings)
- Target status
- Current design is below target; 13 items and 5 raters is the lowest-workload qualifying design.
Assumes a balanced, fully crossed persons x items x raters design with both facets treated as random.
Counts are searched as whole numbers from 1 through 500 without rounding coefficients before target checks.
How to use this calculator
- Enter the person, item, rater, interaction, and residual variance components from the G-study.
- Set the planned number of items and raters for the D-study design you want to evaluate.
- Choose relative G for ranking decisions or absolute Phi for fixed-score decisions.
- Enter the target coefficient, then compare the current design with the optimized item-rater search.
How the D-study is calculated
This calculator uses the standard balanced, fully crossed persons by items by raters D-study model. It assumes the item and rater facets are random and that the variance components have already been estimated in a correctly specified G-study.
Let Vp be person variance, Vi item variance, Vr rater variance, Vpi person-by-item variance, Vpr person-by-rater variance, Vir item-by-rater variance, Ve residual variance, ni items, and nr raters. The relative error variance is:
delta = Vpi / ni + Vpr / nr + Ve / (ni * nr)
The absolute error variance adds the systematic item, rater, and item-by-rater components:
Delta = delta + Vi / ni + Vr / nr + Vir / (ni * nr)
The relative G coefficient is G = Vp / (Vp + delta). The absolute Phi coefficient is Phi = Vp / (Vp + Delta). The relative SEM is sqrt(delta), and the absolute SEM is sqrt(Delta).
Which result to use
Use relative G when the decision is mainly about ordering people, such as ranking examinees or comparing performances. Use absolute Phi when the decision depends on the observed score level itself, such as pass-fail, placement, mastery, or competency decisions. Phi is usually lower because item difficulty, rater severity, and their interaction affect absolute scores even when rank ordering is stable.
What moves reliability most
Adding items mostly reduces person-by-item error and item difficulty effects. Adding raters mostly reduces person-by-rater error and rater severity effects. The best workload choice depends on which variance components are large. The search checks every whole-number design from 1 through 500 items and 1 through 500 raters, then chooses the design that reaches the selected target with the fewest item-rater observations. Ties go to fewer raters, then fewer items.
What this calculator leaves out
The formulas do not estimate variance components from raw scores. They also do not cover nested designs, partially crossed designs, fixed facets, missing ratings, unequal cell sizes, or policy questions about validity and fairness. If an estimated variance component is negative, decide how to treat it in the G-study before entering values here; this calculator accepts only nonnegative components.
Worked example
Using the default values, Vp = 40, Vi = 5, Vr = 2, Vpi = 20, Vpr = 8, Vir = 5, Ve = 20, ni = 10, and nr = 2.
The relative error variance is 20 / 10 + 8 / 2 + 20 / 20 = 7.00. The absolute error variance is 7.00 + 5 / 10 + 2 / 2 + 5 / 20 = 8.75. That gives G = 40 / 47 = 85.11% and Phi = 40 / 48.75 = 82.05%.
For a 90% absolute Phi target, the current 10-item, 2-rater design is below target. Holding items at 10 requires 7 raters. Holding raters at 2 has no finite fixed-facet solution because rater-related absolute error alone is too large. The lowest-workload design within the search range is 13 items by 5 raters.
Common questions
What is the difference between a G-study and a D-study?
A G-study estimates variance components from observed scores. A D-study uses those components to project reliability for a planned measurement design, such as changing the number of items, stations, raters, occasions, or tasks.
Should I use relative G or absolute Phi?
Use relative G when the decision is based on ranking people. Use absolute Phi when the score level matters, such as pass-fail or competency decisions. Phi includes more sources of error, so it should not exceed the relative G coefficient for the same design.
How do I get the variance components?
They come from a generalizability study fitted to raw scores with the same facet structure you plan to generalize over. The components should be reviewed before use, especially if the model produced negative estimates or if the data were unbalanced.
Why can adding items help more than adding raters?
The gain depends on the variance components. If person-by-item variance or item difficulty variance is large, adding items reduces a larger part of the error. If rater-related variance is small, adding more raters may add workload with little improvement.
Does the lowest-workload design also have the lowest cost?
Not necessarily. The search minimizes item-rater observations only. It does not know whether item development, station time, rater training, scheduling, scoring time, or examinee burden is the limiting cost in your setting.