Bifactor Indices · semstudio.tools

What to do

Fit the bifactor model in Mplus, with every item loading on the general factor and on at most one specific factor, and with the factors uncorrelated. If the input has no OUTPUT command, add this line, exactly as it is written here: OUTPUT: STDYX;. If it already has one, add the option STDYX to that command instead of writing a second one.
Run that input in Mplus. The program writes a plain text file with the same name as the input and the extension .out, in the same folder as the input.
Drag the .out file onto the first box below, or click that box, or move the focus to it and press Enter, to choose the file from your computer. It is read inside your browser and semstudio.tools does not transmit the file contents; processing occurs in the browser.
Say which of the factors in the file is the general factor. The page proposes the factor with the most items, because an output does not record which factor was meant to be general; every other factor is then a specific factor. If the output holds several groups, say which group to read as well.
Read the indices in the three tables, at the level of the model, of each factor and of each item, and copy the same values as tab-separated text from the block at the end.
For relative parameter bias only: fit the same items as a single-factor model, with the same OUTPUT: STDYX; line, and drop that second .out file on the second box. The page matches the items by name and reports the bias per item. Without that file the two bias columns stay empty.

Inputs are not analyzed when the STDYX Standardization section is absent, an indicator is itself latent, factors are correlated, an item name is repeated under one BY statement, or the model contains only one factor. The page also requires a recognizable Mplus output. All numerical inputs are read from the file.

Load the Mplus output of the bifactor model

Drop the Mplus .out file of the bifactor model here, or press Enter to choose one.

The file is read in your browser. Windows-1252, UTF-8 and UTF-16 output files are all decoded. When the output has more than one group the page asks which of them to read.

Load the unidimensional model as well, for relative parameter bias

Relative parameter bias compares the loading an item takes in a single-factor model with the loading it takes on the general factor of the bifactor model. The first of those is in a different run, so it comes from a second output file. Fit the same items as one factor, with OUTPUT: STDYX; in the input, and drop the resulting .out file here. Everything else on the page is computed without it.

Drop the Mplus .out file of the unidimensional model here, or press Enter to choose one.

The items are matched by name to those of the bifactor model. An item that is in one file and not in the other is named and left without a bias.

Results

The indices appear here once the output of a bifactor model has been read.

Copy-ready table

The block below repeats every result as tab-separated text, so that it can be moved into a spreadsheet or into a manuscript table without retyping.

Results as tab-separated text

Reading the indices

The values below are guidance taken from the sources named, not decision rules. None of them replaces an evaluation of the model as a whole, and each was proposed for a particular family of measures and simulation conditions.

Rodriguez, Reise and Haviland (2016b) treat an explained common variance above about .70 together with a percentage of uncontaminated correlations above about .70 as pointing to a general factor strong enough that fitting a unidimensional model to the same data would distort the structural parameters only slightly.
Reise, Scheines, Widaman and Haviland (2013) discuss the joint role of the three quantities: when the percentage of uncontaminated correlations is below about .80, they read a general explained common variance above about .60 together with an omega hierarchical above about .70 as multidimensionality that is not severe enough to prevent interpreting the instrument as primarily unidimensional.
Bonifay, Reise, Scheines and Meijer (2015) relate the explained common variance to the size of the bias directly: in their conditions relative bias fell below 10% once the explained common variance exceeded about .70, and below 5% once it exceeded about .80.
Hancock and Mueller (2001) define construct replicability H as the squared correlation between a factor and the optimally weighted composite of its indicators; values around .80 are discussed as an indication that the factor is well defined by the items measuring it.
Gorsuch (1983) discusses .90 as a reference value for factor determinacy; below this level, substituting estimated factor scores for the latent factor may be difficult to justify.
A low omega hierarchical subscale value indicates that little reliable subscale variance remains attributable to the specific factor after accounting for the general factor (Reise, Bonifay, & Haviland, 2013).

These indices describe a fitted bifactor solution; they do not test whether the bifactor structure generated the data. Bonifay and Cai (2017) show that bifactor models have high fitting propensity, so global fit alone provides limited evidence for the structure. Bonifay, Lane and Reise (2017) make the same point for psychopathology models and note that specific factors with little reliable variance can be difficult to interpret.

Methods and formulas

The indices are those of Rodriguez, Reise and Haviland (2016a), computed as in the Bifactor Indices Calculator of Dueber (2017). Equations 1, 4 and 7 of Rodriguez, Reise and Haviland (2016b) are used in the form their published correction gives (Journal of Personality Assessment, 98(4), 444): in omega the second term of the numerator sums the squared sums of the specific loadings and not of the general ones; in omega hierarchical subscale the denominator carries the squared sum of the general loadings of the subscale and the squared sum of the loadings on that specific factor; and in the explained common variance the denominator carries the sum, over the specific factors, of the sums of the squared loadings. A value computed from the equations as they were first printed differs from the value reported here, and for the last two it still looks like a coefficient: on a nine-item model with three specific factors, the first printed form gives 3.217 in place of an omega of .890, the second .144 in place of an omega hierarchical subscale of .201, and the third .735 in place of an explained common variance of .755. Equations 1, 4 and 7 of Rodriguez, Reise and Haviland (2016b) are used in the form their published correction gives (Journal of Personality Assessment, 98(4), 444): in omega the second term of the numerator sums the squared sums of the specific loadings and not of the general ones, in omega hierarchical subscale the denominator carries the squared sum of the general loadings and the squared sum of the loadings on that specific factor, and in the explained common variance the denominator carries the sum over the specific factors of the sums of the squared loadings. A value computed from the equations as they were first printed differs from the value reported here, and for omega hierarchical subscale and for the explained common variance it still looks like a coefficient. Write λ_ik for the standardized loading of item i on factor k, with k = G for the general factor and k = 1 … S for the specific factors. An item is treated as loading on a factor when that loading is present in the output and different from zero. The uniqueness of item i, that is its standardized residual variance, is

θ_{i} = 1 - \sum_{k} {λ_{i k}}^{2}

Here θ_i is the uniqueness of item i, and the sum runs over the general factor and over every specific factor the item loads on, λ_ik being its standardized loading on factor k.

When the standardized residual variances are printed in the output for every item, those values are used instead of this identity. A negative uniqueness is evaluated against the rounding error of the printed output rather than against zero: the loadings are printed to three decimals, so the sum of their squares carries an error of at most $0.001 \sum_{j} | λ_{i j} |$ , and a value negative by no more than that is set to zero and reported. A value more negative than the rounding can account for is a Heywood case: the common factors account for more than the whole variance of the item, the solution is inadmissible, and no index is computed from it.

Omega and omega hierarchical

For a factor f, let I_f be the set of items loading on it. Omega is the proportion of the variance of the unit-weighted total of those items that is accounted for by all the common factors together:

ω_{f} = \frac{\sum_{k} {(\sum_{i} λ_{i k})}^{2}}{\sum_{i} θ_{i} + \sum_{k} {(\sum_{i} λ_{i k})}^{2}}

Here ω_f is the omega of the total of the items of factor f, k runs over every factor of the model, i runs over the items loading on f, λ_ik is the standardized loading of item i on factor k, and θ_i the uniqueness of item i.

Omega hierarchical keeps the same denominator and retains in the numerator only the factor of interest, so it is the proportion of the variance of that total attributable to one factor. For the general factor,

ω_{H} = \frac{{(\sum_{i} λ_{i G})}^{2}}{\sum_{i} θ_{i} + \sum_{k} {(\sum_{i} λ_{i k})}^{2}}

Here ω_H is omega hierarchical, i runs over the items loading on the general factor, λ_iG is the loading of item i on it, θ_i the uniqueness of item i, and the sum over k in the denominator runs over every factor as before.

For a specific factor s the same ratio is taken over the items of its own subscale:

ω_{HS} (s) = \frac{{(\sum_{i} λ_{i s})}^{2}}{\sum_{i} θ_{i} + \sum_{k} {(\sum_{i} λ_{i k})}^{2}}

Here ω_HS(s) is omega hierarchical subscale for the specific factor s, i runs over the items of that subscale, and λ_is is the loading of item i on s.

For the general factor the items are all of them, so ω is the omega of the whole scale and ω_H the part of it carried by the general factor; for a specific factor the items are those of the subscale, so ω is the omega of the subscale score and ω_HS the part of it left once the general factor is partialed out. The general-factor omega is the omega of a multidimensional composite in the sense of Zinbarg, Revelle, Yovel and Li (2005). Relative omega is their ratio:

relative ω_{f} = \frac{ω_{H} (f)}{ω_{f}}

Here ω_H(f) is omega hierarchical for factor f, which for a specific factor is ω_HS, and ω_f is the omega of the same total.

It is the share of the reliable variance of the composite that belongs to the factor of interest.

Every coefficient on this page is derived for a bifactor model whose residuals are uncorrelated. The covariance that a subset of items share beyond the general factor is what the specific factor of that subset stands for, so a residual covariance estimated alongside it is confounded with that factor, and Rodriguez, Reise and Haviland (2016a) write every index without one. An output that estimates such a covariance is refused with its name, rather than read into a formula that does not cover it. A shared method or a common wording belongs in a model of this kind as a further specific factor, orthogonal to the others; the reliability of a congeneric factor whose model does carry residual covariances is computed on the reliability page, where that coefficient is defined.

Explained common variance

The common variance of item i is $\sum_{k} {λ_{i k}}^{2}$ , and the explained common variance of a factor is its share of a sum of such quantities. Two denominators are in use. Over all N items,

{ECV}_{f} = \frac{\sum_{i = 1}^{N} {λ_{i f}}^{2}}{\sum_{i = 1}^{N} \sum_{k} {λ_{i k}}^{2}}

Here ECV_f is the explained common variance of factor f, N the number of items in the instrument, λ_if the loading of item i on f, and the inner sum over k runs over every factor.

For the general factor this is the quantity usually called ECV (Reise, Scheines, Widaman & Haviland, 2013). For a specific factor it is what Stucky and Edelen (2015) call the specific-dimension ECV, labeled ECV_SG in the calculator of Dueber (2017): the strength of that specific factor against all the common variance in the instrument, including the items that do not load on it. Restricting both sums to the items of the subscale gives the second form, Dueber's ECV_SS:

{ECV}_{SS} (s) = \frac{\sum_{i} {λ_{i s}}^{2}}{\sum_{i} \sum_{k} {λ_{i k}}^{2}}

Here ECV_SS(s) is the share of the common variance within the subscale of the specific factor s that belongs to s itself; both sums over i run over the items of that subscale, and the inner sum over k runs over every factor.

This is the strength of the specific factor within its own items. The share the general factor takes within the same items, the within-domain ECV of Stucky and Edelen (2015), is reported alongside it:

{ECV}_{GS} (s) = \frac{\sum_{i} {λ_{i G}}^{2}}{\sum_{i} \sum_{k} {λ_{i k}}^{2}}

Here ECV_GS(s) is the share of the same common variance that belongs to the general factor, λ_iG being the loading of item i on it; the sums over i run over the items of the subscale as before.

ECV_GS and ECV_SS add to one when the items of the subscale load on the general factor and on that specific factor only; when some of them also load on another specific factor the two shares add to less than one, and the remainder belongs to the other specific factors. The same ratio taken one item at a time is the item explained common variance of Stucky, Thissen and Edelen (2013):

{I-ECV}_{i} = \frac{{λ_{i G}}^{2}}{\sum_{k} {λ_{i k}}^{2}}

Here I-ECV_i is the share of the common variance of item i alone that belongs to the general factor, and the sum runs over every factor the item loads on.

Percentage of uncontaminated correlations

A correlation between two items is contaminated, in the sense of Reise, Scheines, Widaman and Haviland (2013), when both items load on the same specific factor, since then the covariance between them carries specific variance as well as general variance. With a clean cluster structure, in which every item loads on the general factor and on at most one specific factor, the contaminated pairs can be counted subscale by subscale:

PUC = 1 - \frac{\sum_{s} \frac{n_{s} (n_{s} - 1)}{2}}{\frac{N (N - 1)}{2}}

Here PUC is the percentage of uncontaminated correlations expressed as a proportion, s runs over the specific factors, n_s is the number of items in subscale s, and N is the number of items in the instrument.

Items that load on the general factor only enter N and belong to no subscale, which is what the formula requires. When an item loads on two or more specific factors the subscale-by-subscale count adds the same pair more than once and the formula returns too small a value, so a generalized version is computed as well, counting each pair of items that share at least one specific factor exactly once:

{PUC}^{'} = 1 - \frac{C}{\frac{N (N - 1)}{2}}

Here PUC′ is that generalized value and C is the number of pairs of items i < j that share at least one specific factor, each pair counted once however many factors it shares.

The two agree exactly whenever no item loads on more than one specific factor. The page reports both and states which one applies to the pattern the file holds.

Construct replicability

H is the squared correlation between a factor and the optimally weighted composite of its indicators (Hancock & Mueller, 2001). With orthogonal factors and standardized loadings it reduces to

H_{f} = \frac{1}{1 + \frac{1}{\sum_{i} [\frac{{λ_{i f}}^{2}}{1 - {λ_{i f}}^{2}}]}}

Here H_f is the construct replicability of factor f, the sum over i runs over the items loading on it, and λ_if is the standardized loading of item i on that factor.

It is computed here for the general factor over all items and for each specific factor over the items of its subscale. A standardized loading of exactly one makes the term undefined and is reported rather than computed.

Factor determinacy

Factor determinacy is the correlation between a factor and the factor score estimate obtained for it (Gorsuch, 1983). Collecting the loadings in the N × (S + 1) matrix Λ and the uniquenesses in the diagonal matrix Θ, the model-reproduced correlation matrix under orthogonal factors is

Σ = Λ Λ^{'} + Θ

Here Σ is the reproduced correlation matrix of the items, Λ the matrix of standardized loadings with one row per item and one column per factor, Λ′ its transpose, and Θ the diagonal matrix of uniquenesses.

The determinacies are the square roots of the diagonal of

Λ^{'} Σ^{- 1} Λ

Here Σ⁻¹ is the inverse of the reproduced correlation matrix, and the diagonal of the product holds one entry per factor, in the order of the columns of Λ.

This form assumes that the general factor and the specific factors are mutually orthogonal, which is the usual bifactor specification and the assumption under which every other index on this page is also computed. The square of the determinacy is reported as well, since it is the proportion of the variance of the factor score estimate that the factor accounts for.

Σ is factorized by Cholesky decomposition, which succeeds only if Σ is positive definite, and the linear systems $Σ X = Λ$ are solved by forward and back substitution rather than by forming the inverse. When the factorization fails, the page reports that Σ is not positive definite, names the item at which the factorization broke down, and reports no determinacy at all instead of a number obtained from a fallback that would not be meaningful. A matrix that is only nearly singular is detected the same way through the size of the pivots relative to the largest diagonal entry.

Relative parameter bias

When the output of the unidimensional model for the same items has been loaded as well, the relative parameter bias of item i is the difference between the loading in the unidimensional solution and the loading on the general factor in the bifactor solution, divided by the latter:

{RPB}_{i} = \frac{λ_{i, uni} - λ_{i G}}{λ_{i G}}

Here RPB_i is the relative parameter bias of item i, λ_i,uni its standardized loading in the single-factor model read from the second file, and λ_iG its standardized loading on the general factor of the bifactor model.

A positive value therefore means that the unidimensional model gives the item a larger loading than the bifactor model does. This is the order used by Rodriguez, Reise and Haviland (2016b); the Bifactor Indices Calculator of Dueber (2017) takes the difference the other way around, so the values printed here have the opposite sign to the ones it prints, and the same absolute value. The average absolute relative parameter bias is the mean of |RPB_i| over the items for which a unidimensional loading was read.

What the page does not do

Every formula above assumes orthogonal factors, continuous indicators on a standardized metric, and one general factor. Correlated specific factors, second-order and higher-order models, two-tier models with more than one general factor, and the extra terms that categorical indicators bring to the reliability of an observed total score are outside what is computed here. With categorical indicators the loadings read from the STDYX section refer to the latent response variables, so the indices describe the model on that metric and not the metric of the observed responses.

Three structures are refused rather than computed. An output whose indicators include a latent variable is a second-order or hierarchical model, and none of the formulas above applies to it; the model has to be respecified and fitted as a bifactor model, and that run loaded instead. An output in which two factors have a non-zero correlation, read from the WITH lines of the STDYX section, violates the orthogonality every formula assumes. An output that prints the same item name twice under one BY heading would lose one of the two loadings when the table is built. Two further cases are computed with a message. An output with several groups holds a separate solution for each of them, and the page asks which group to read instead of taking the first. A specific factor with a single indicator is computed, because the arithmetic is defined, with a note that its indices describe one item rather than a factor.

References

Bonifay, W., & Cai, L. (2017). On the complexity of item response theory models. Multivariate Behavioral Research, 52(4), 465–484. https://doi.org/10.1080/00273171.2017.1309262
Bonifay, W., Lane, S. P., & Reise, S. P. (2017). Three concerns with applying a bifactor model as a structure of psychopathology. Clinical Psychological Science, 5(1), 184–186. https://doi.org/10.1177/2167702616657069
Bonifay, W. E., Reise, S. P., Scheines, R., & Meijer, R. R. (2015). When are multidimensional data unidimensional enough for structural equation modeling? An evaluation of the DETECT multidimensionality index. Structural Equation Modeling, 22(4), 504–516. https://doi.org/10.1080/10705511.2014.938596
Dueber, D. M. (2017). Bifactor Indices Calculator: A Microsoft Excel-based tool to calculate various indices relevant to bifactor CFA models [Computer software]. https://doi.org/10.13023/edp.tool.01
Gorsuch, R. L. (1983). Factor analysis (2nd ed.). Erlbaum.
Hancock, G. R., & Mueller, R. O. (2001). Rethinking construct reliability within latent variable systems. In R. Cudeck, S. du Toit, & D. Sörbom (Eds.), Structural equation modeling: Present and future (pp. 195–216). Scientific Software International.
Reise, S. P., Bonifay, W. E., & Haviland, M. G. (2013). Scoring and modeling psychological measures in the presence of multidimensionality. Journal of Personality Assessment, 95(2), 129–140. https://doi.org/10.1080/00223891.2012.725437
Reise, S. P., Scheines, R., Widaman, K. F., & Haviland, M. G. (2013). Multidimensionality and structural coefficient bias in structural equation modeling: A bifactor perspective. Educational and Psychological Measurement, 73(1), 5–26. https://doi.org/10.1177/0013164412449831
Rodriguez, A., Reise, S. P., & Haviland, M. G. (2016a). Evaluating bifactor models: Calculating and interpreting statistical indices. Psychological Methods, 21(2), 137–150. https://doi.org/10.1037/met0000045
Rodriguez, A., Reise, S. P., & Haviland, M. G. (2016b). Applying bifactor statistical indices in the evaluation of psychological measures. Journal of Personality Assessment, 98(3), 223–237. https://doi.org/10.1080/00223891.2015.1089249 Correction: Journal of Personality Assessment, 98(4), 444. https://doi.org/10.1080/00223891.2015.1117928
Stucky, B. D., & Edelen, M. O. (2015). Using hierarchical IRT models to create unidimensional measures from multidimensional data. In S. P. Reise & D. A. Revicki (Eds.), Handbook of item response theory modeling: Applications to typical performance assessment (pp. 183–206). Routledge.
Stucky, B. D., Thissen, D., & Edelen, M. O. (2013). Using logistic approximations of marginal trace lines to develop short assessments. Applied Psychological Measurement, 37(1), 41–57. https://doi.org/10.1177/0146621612462759
Zinbarg, R. E., Revelle, W., Yovel, I., & Li, W. (2005). Cronbach's α, Revelle's β, and McDonald's ω_H: Their relations with each other and two alternative conceptualizations of reliability. Psychometrika, 70(1), 123–133. https://doi.org/10.1007/s11336-003-0974-7