Model Fit and Local Fit

What the page reads

The whole of MODEL FIT INFORMATION, the estimator and the counts in SUMMARY OF ANALYSIS, the title of the run, and, when they are there, the residual output and the modification indices. A file that carries none of the fit information is refused and named. Nothing in the table is recomputed from the raw data, because the raw data are not in the file.

The one index that is added

Mplus prints the chi-square test, the RMSEA with its confidence interval and the probability of close fit, the CFI and the TLI, and the standardized root mean square residual. The table adds McDonald's (1989) centrality index, which is a function of the same discrepancy:

Here $F_0$ is the population discrepancy used for Mc, N the sample size and df the degrees of freedom of the model. Mc is on a zero to one scale like the CFI but, unlike it, does not depend on a baseline model, which is why it is worth having next to the CFI when the baseline model of a run is itself unusual. For the separate consistency check against the RMSEA printed by Mplus, the page follows the Mplus convention exactly: $\mathrm{RMSEA}=\sqrt{\frac{\max ({\chi }^2-\mathrm{df},0)}{N\mathrm{df}}}\sqrt{G}$, where N is the total sample size and G is the number of groups. Mplus uses N, rather than N−1, in this calculation. A disagreement with an unscaled value is therefore reported rather than silently ignored; for scaled statistics, a discrepancy can instead reflect the estimator-specific correction.

The differences between models

The table gives the difference in CFI, in RMSEA and in SRMR against the model chosen as the reference, and the difference in chi-square and in degrees of freedom. No cutoff is attached to these differences here. Two reasons: the cutoffs in circulation were derived for measurement invariance, where they belong to the invariance page, which applies them with the conditions of the studies that produced them; and a difference in chi-square between models estimated with a scaled statistic is not the difference of the printed values, but the corrected quantity computed on the difference test page. When the estimator of a file produces a scaled chi-square, the table says so on that row, so that the difference is not read off it by mistake.

Akaike weights over the models that were loaded

Two models that are not nested cannot be compared by a difference test, and the Akaike criterion is what is usually looked at instead. The difference between two of its values is hard to read on its own, so it is put on a scale. With $D_i\left(\mathrm{AIC}\right)={\mathrm{AIC}}_i-\min \left(\mathrm{AIC}\right)$ the weight of a model is

which is Equation 4 of Wagenmakers and Farrell (2004), their Equation 2 being the difference above. The weights add to one over the set of models considered, and the ratio of two of them is the evidence ratio, which says how many times better supported one model is than another. Two things follow from the construction and are worth keeping in view: the weights are conditional on the set of models loaded on this page and change when a model is added or taken away, and a comparison by an information criterion is a comparison between models fitted to the same observations, which the page checks and says when the counts differ.

The conventional cutoffs

When enabled, the table marks CFI ≥ .95, RMSEA ≤ .06 and SRMR ≤ .08, the values examined by Hu and Bentler (1999) under specific simulation conditions. These values are reference points rather than universal decision rules. Their behavior varies with model, sample and misspecification (Marsh, Hau, & Wen, 2004), so they are reported alongside local-fit diagnostics rather than used as a stand-alone verdict.

The residuals

The residual output prints the model estimated matrix and the residual matrix, observed minus estimated, and, for the estimators that admit it, the standardized residual, which is the residual divided by its own standard error and is read as a z score. The page orders them by absolute size, which the output does not, and counts those beyond 1.96 and beyond 2.58; values of 999 are the Mplus sign for a quantity that is not defined and are dropped rather than shown.

It also puts the residuals in the metric of a correlation, which is the metric in which they are read. The observed matrix is the sum of the two printed matrices, so with $s_{ij}={\hat{\sigma }}_{ij}+e_{ij}$ the correlation residual is

The two matrices are standardized and subtracted. The result is on the scale of a correlation: a correlation residual of .12 says the model reproduces that correlation .12 too low, whatever the sample size, whereas a z score of 3.4 says only that the discrepancy is large relative to its standard error, which in a large sample it will be even when the discrepancy is trivial. No cutoff is attached to it here.

The modification indices

A modification index is the score test of the hypothesis that one fixed parameter is zero: it is what the chi-square of the model would fall by, approximately, if that parameter alone were freed, and it is distributed as a chi-square with one degree of freedom under the hypothesis (Sörbom, 1989). The expected parameter change next to it is the value the freed parameter would take, and the standardized one is that value in the metric of the standardized solution. The page orders all of them together by size, which the output does not do, since Mplus prints them grouped by type of statement.

When the file was produced with MODINDICES(0), the page reports the ordinary χ²(1) .05 critical value (3.84) and a Bonferroni-adjusted value, χ²₁(1 − .05/m), where m is the number of tested modifications (Dunn, 1961). If Mplus printed only indices above a positive threshold, the full family size is unknown and the adjusted value is omitted. The correction addresses multiplicity; it does not provide a substantive rationale for freeing a parameter (MacCallum, Roznowski, & Necowitz, 1992).

The power of each test, and the reading that follows from it

The modification index is a statistical test, so its power depends on the sample size and on the size of the deviation it is asked to find. In a large sample a deviation of no substantive size gives a significant index; in a small one a real deviation gives an index that is not significant. Saris, Satorra and van der Veld (2009) show that both readings can be settled with quantities the output already carries. The relation between the index, the expected parameter change and the standard error of that change is

their Equations 6 and 7. The standard error gives a confidence interval for the parameter that is being tested, $\mathrm{EPC}\mp 1.96\sigma$ (Eq. 8), which the table reports: it says how precisely the size of the deviation is being estimated, which the index alone does not.

It also gives the power. Let δ be the smallest deviation one would want the test to find. Under a deviation of that size the index is distributed as a noncentral chi-square with one degree of freedom and noncentrality parameter

their Equations 9 and 10, and the power of the test is $\mathrm{Prob}\left({\chi }^2\right(1,\mathrm{ncp})>c_{\alpha })$, with $c_{\alpha }$ the critical value of the test with one degree of freedom (Eq. 11). The page computes it with the same noncentral chi-square routine the power page uses. Because δ and the expected parameter change have to be in one metric, the StdYX change is the one used.

Saris, Satorra and van der Veld combine the modification-index test with its power. A significant index is interpreted together with the expected parameter change and the target deviation δ; a non-significant index is informative only when the test had adequate power for δ. Low-power non-significance remains inconclusive. The page reports these components separately rather than treating significance alone as evidence of misspecification.

The values above the table are the ones the authors suggest, and they say of them that they are arbitrary and belong to the standards of a discipline rather than to statistics: .1 for a standardized structural parameter or a residual covariance, .4 for a loading, and .75 for the power above which a test counts as sensitive. They are meant to be set before the output is read.

Self-test

Append ?selftest=1 to check fit parsing, RMSEA and Mc, residual matrices, modification indices, EPC standard errors, critical values, power calculations, and unsupported inputs.

• Browne, M. W., & Cudeck, R. (1992). Alternative ways of assessing model fit. Sociological Methods & Research, 21(2), 230–258. https://doi.org/10.1177/0049124192021002005
• Dunn, O. J. (1961). Multiple comparisons among means. Journal of the American Statistical Association, 56(293), 52–64. https://doi.org/10.1080/01621459.1961.10482090
• Wagenmakers, E.-J., & Farrell, S. (2004). AIC model selection using Akaike weights. Psychonomic Bulletin & Review, 11(1), 192–196. https://doi.org/10.3758/BF03206482
• Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. https://doi.org/10.1080/10705519909540118
• MacCallum, R. C., Roznowski, M., & Necowitz, L. B. (1992). Model modifications in covariance structure analysis: The problem of capitalization on chance. Psychological Bulletin, 111(3), 490–504. https://doi.org/10.1037/0033-2909.111.3.490
• Marsh, H. W., Hau, K.-T., & Wen, Z. (2004). In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler's (1999) findings. Structural Equation Modeling, 11(3), 320–341. https://doi.org/10.1207/s15328007sem1103_2
• McDonald, R. P. (1989). An index of goodness-of-fit based on noncentrality. Journal of Classification, 6(1), 97–103. https://doi.org/10.1007/BF01908590
• Saris, W. E., Satorra, A., & van der Veld, W. M. (2009). Testing structural equation models or detection of misspecifications? Structural Equation Modeling, 16(4), 561–582. https://doi.org/10.1080/10705510903203433
• Steiger, J. H., & Lind, J. C. (1980, May). Statistically based tests for the number of common factors [Paper presentation]. Annual meeting of the Psychometric Society, Iowa City, IA, United States.
• Sörbom, D. (1989). Model modification. Psychometrika, 54(3), 371–384. https://doi.org/10.1007/BF02294623