Reliability Coefficients

1. Open the Mplus input file of your confirmatory factor analysis, the file with the extension .inp. If it has no OUTPUT command, add this line, exactly as it is written here: OUTPUT: SAMPSTAT STDYX;. If it already has one, add the two options SAMPSTAT STDYX to that command instead of writing a second one.
2. Run that input in Mplus. The program writes a plain text file with the same name as the input and the extension .out, in the same folder as the input.
3. Drag the .out file onto the box below, or press Choose the .out file and pick it there. The file is read inside your browser and semstudio.tools does not transmit the file contents; processing occurs in the browser.
4. Wait for the list headed What the file contained. It names the estimator, the factors, the items of each factor and the sections the coefficients need, one line each.
5. Read the coefficients in the table printed for each factor, and the summary table under them.
6. Omega itself is computed here and needs nothing further. If a standard error and a confidence interval for it are wanted as well, and the indicators are continuous, copy the MODEL CONSTRAINT syntax printed under the tables and run it in Mplus: that interval is the one thing an output does not carry.

The page requires a recognizable Mplus output, a MODEL RESULTS section, and an estimator reported in SUMMARY OF ANALYSIS. A factor is omitted when its scale cannot be reconstructed or its solution is inadmissible. The message identifies the missing or unsupported information and, where relevant, the OUTPUT option needed to obtain it.

Omega with continuous indicators

Omega (McDonald, 1999) is the reliability of the sum of the items implied by the estimated one-factor measurement model. The same quantity is reported in the structural equation modeling literature as composite reliability, written ρ_c and often attributed to Fornell and Larcker (1981), and, with uncorrelated residuals, as coefficient omega, Raykov's rho and construct reliability; the names differ, the formula does not. It is computed from the unstandardized parameters printed under MODEL RESULTS:

Here λ_i is the unstandardized loading of item i on the factor, ψ is the variance of the factor, θ_ii is the residual variance of item i, and θ_ij is the residual covariance between items i and j, summed over the pairs with i < j and counted twice because it enters the variance of the sum twice.

ψ is read from the output: it is 1.000 when the factor is identified by fixing its variance, and it is the value printed under Variances when the marker-variable method is used. For an endogenous factor Mplus prints the residual variance instead, and the total variance is recovered as $\psi =\theta /(1-R^2)$ from the R-SQUARE section; when R-SQUARE is absent the page says so rather than substituting a value. A residual covariance between items of the same factor belongs to the total variance of that sum; one between items of different factors belongs to neither sum and enters no coefficient.

Average variance extracted

Here k is the number of items and λ*_i the completely standardized loading of item i, read from the STDYX solution.

AVE (Fornell & Larcker, 1981) is the average share of an item's variance that the factor accounts for. On a standardized solution the residual variance of an item is 1 − λ*_i², so the ratio of the common part to the total reduces to the mean of the squared standardized loadings, which is the form above. The same ratio computed from unstandardized loadings and residual variances is a different quantity: it weights each item by its variance, so an item measured on a wider scale counts for more, and it is not what Fornell and Larcker define. AVE is a property of the individual items and omega is the reliability of their sum, so the two coefficients answer different questions and neither is a bound on the other.

Cronbach's alpha

Alpha (Cronbach, 1951) is computed from the estimated covariance matrix that SAMPSTAT prints, restricted to the items of the factor:

Here k is the number of items of the factor, σ_ii is the variance of item i from that matrix, and σ²_sum is the variance of the total, that is the sum of every element of the submatrix, diagonal and off-diagonal alike.

Alpha is the value reliability takes when the items are tau-equivalent, that is when their unstandardized loadings are equal, and their residuals are uncorrelated. When tau-equivalence does not hold and the residuals are uncorrelated, alpha is a lower bound. When the residuals are correlated, or the one-factor model is misspecified in some other way, alpha can also exceed the reliability of the sum, so the direction of the discrepancy is not fixed in advance (Cho, 2016; Flora, 2020). Omega is the coefficient that matches the estimated model, on the condition that the model itself fits. The page prints the ratio between the largest and the smallest loading in absolute value as a description of how far the loadings are from equal; the formal comparison is a chi-square difference test between the congeneric and the tau-equivalent model.

Ordered categorical indicators

With CATEGORICAL ARE and a weighted least squares estimator, the loadings refer to the underlying response variables y*, not to the observed responses. A star marks a quantity on that metric. Omega on it uses the standardized loadings:

Here λ*_i is the standardized loading of item i on the y* metric, θ*_ii is its residual variance on that metric, and θ*_ij is the residual correlation between items i and j, summed over the pairs with i < j.

Where the standardized solution carries no residual variances, which is the usual case under the delta parameterization, ${\theta }_{ii}^{\ast }=1-{\lambda }_i^{\ast 2}$ is used. This coefficient describes the sum of the y*, a score that is not observed.

The average variance extracted on the same metric follows from that identity, which makes the denominator equal to the number of items:

The symbols are those of the previous formula, with k the number of items of the factor.

Ordinal alpha (Zumbo, Gadermann & Zeisser, 2007) is alpha applied to the polychoric correlation matrix of the items, taken from SAMPSTAT:

Here k is the number of items and ρ_ij is the polychoric correlation between items i and j, the double sum running over every element of the matrix, so that the diagonal contributes k.

Ordinal alpha is computed directly from the polychoric correlation matrix. Zumbo, Gadermann and Zeisser (2007, Eq. 1) give an equivalent one-factor expression only when that factor reproduces the matrix. The coefficient is reported only when the assembled polychoric matrix is positive definite. It is defined on the latent-response (y*) metric and is not the reliability of the observed category sum. It often exceeds Pearson-based alpha, but that ordering is not guaranteed.

Green and Yang omega

The Green and Yang (2009) coefficient is the reliability of the sum of the observed ordinal responses. It reconstructs the variances and covariances of the observed items from the thresholds and from the correlations between the underlying variables, using the bivariate normal distribution function:

Here X_i is the observed response to item i, c and d run over the thresholds of items i and j, τ_ic is the c-th threshold of item i, ρ_ij is the correlation between the two underlying variables, Φ₂ is the bivariate standard normal distribution function and Φ the univariate one.

The two correlations are not the same. In the numerator, which is the variance of the sum attributable to the common factor, ${\rho }_{ij}={\lambda }_i^{\ast }{\lambda }_j^{\ast }$, and on the diagonal ${\rho }_{ii}={\lambda }_i^{\ast 2}$. In the denominator, which is the variance of the observed sum, ρ_ij is the polychoric correlation itself, taken from the sample matrix that SAMPSTAT prints, with 1 on the diagonal; this is the denominator of Green and Yang (2009, Eq. 21). Building it from the sample matrix rather than from the one the model reproduces means the coefficient carries the misfit of the model with it, which is what its authors intend: the denominator is the variance of the score as it was observed, not the variance the model would imply for it. The coefficient is left uncomputed when the polychoric matrix is absent from the file, when it does not carry every item of the factor, or when the submatrix of those items is not positive definite.

Here Cov_common is reconstructed from the correlations implied by the common factor alone and Cov_total from the sample polychoric correlations; each double sum runs over every pair of items and over the diagonal.

The diagonal of the denominator is the variance of the observed item, obtained from its thresholds alone. For an item scored 1 to C with thresholds τ₁ to τ_C−1, write $q_c=1-\mathrm{\Phi }({\tau }_c)$ for the probability of a response above the c-th threshold. Then

Here E(X) is the mean of the observed response, c runs over the C − 1 thresholds of the item and q_c is the probability just defined.

Here Var(X) is the variance of the observed response and c and q_c are as above.

This is the same quantity the double sum over thresholds returns with ρ = 1, and is used because it needs no quadrature.

Φ₂ is evaluated from the one-dimensional integral representation of the bivariate normal distribution function, with a 24-node Gauss–Legendre quadrature. The integral is first mapped by the substitution $t=\sin \theta$: in the direct form the factor $1/\sqrt{1-t^2}$ has an integrable singularity at $t=\pm 1$ that polynomial quadrature approximates poorly, and the case of two equal arguments, which is exactly the diagonal of the numerator, is the one where the error is largest. After the substitution the integrand is smooth, and for |ρ| close to 1 the interval is split into subintervals that are packed geometrically toward the endpoint.

The formula assumes that the responses arise from normally distributed underlying variables, so it applies to a probit link. Mplus reports the link on the Link line of SUMMARY OF ANALYSIS. Weighted least squares estimators use a probit link. Maximum likelihood with categorical items uses a logit link unless LINK = PROBIT is requested; with the logit link the thresholds are on the logistic metric and this page does not compute the coefficient.

Confidence intervals

The coefficients on this page are point estimates computed from the parameter estimates the output carries, and nothing has to be run again to obtain them. An interval around omega is a different matter: its standard error depends on the sampling covariance of the loadings and the residual variances, which an output does not carry, so it can only come from a run in which omega is itself a parameter of the model, defined with MODEL CONSTRAINT and asked for with CINTERVAL (Raykov, 1997). That syntax, and only that syntax, is what this page writes for the factors and items of the file loaded. Point estimates of λ and θ are the same under ML and MLR; what changes with MLR is the standard errors, and therefore the intervals.

What these numbers do not settle

Values near .70 are often quoted as a minimum for research on group means, and .80 or higher for decisions about individuals; these are conventions of the literature and are used here only as rough guidance, not as a threshold a scale passes or fails (Cho, 2016; Flora, 2020). A reliability coefficient does not by itself establish that a scale is adequate: it is conditional on the model that produced it, on the sample it was estimated in, and on the use the score is put to, and it says nothing about the validity of the interpretation. A coefficient computed from an inadmissible solution is not a reliability at all, and the page marks those cases instead of reporting the number.

Two coefficients cannot be recovered from an Mplus output with categorical items: the classical alpha on Pearson correlations between the observed responses, and the omega variants that use the observed variance of the total score. Both need the raw data.

• Cho, E. (2016). Making reliability reliable: A systematic approach to reliability coefficients. Organizational Research Methods, 19(4), 651–682. https://doi.org/10.1177/1094428116656239
• Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334. https://doi.org/10.1007/BF02310555
• Flora, D. B. (2020). Your coefficient alpha is probably wrong, but which coefficient omega is right? A tutorial on using R to obtain better reliability estimates. Advances in Methods and Practices in Psychological Science, 3(4), 484–501. https://doi.org/10.1177/2515245920951747
• Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39–50. https://doi.org/10.1177/002224378101800104
• Green, S. B., & Yang, Y. (2009). Reliability of summed item scores using structural equation modeling: An alternative to coefficient alpha. Psychometrika, 74(1), 155–167. https://doi.org/10.1007/s11336-008-9099-3
• McDonald, R. P. (1999). Test theory: A unified treatment. Erlbaum.
• Raykov, T. (1997). Estimation of composite reliability for congeneric measures. Applied Psychological Measurement, 21(2), 173–184. https://doi.org/10.1177/01466216970212006
• Zumbo, B. D., Gadermann, A. M., & Zeisser, C. (2007). Ordinal versions of coefficients alpha and theta for Likert rating scales. Journal of Modern Applied Statistical Methods, 6(1), 21–29. https://doi.org/10.22237/jmasm/1177992180