| Peer-Reviewed

Estimating the Context Effect in a Multilevel Latent Model with Small Sample Sizes: A Monte Carlo Simulation Study

Received: 3 August 2017     Accepted: 11 August 2017     Published: 4 September 2017
Views:       Downloads:
Abstract

In multilevel modeling, the relationships between the criterion and predictors are investigated at different levels. Often, the cluster-level predictors are measured by aggregating the individual-level measures. However, the aggregated cluster-level predictors do not always reliably measure the cluster-level regression coefficient, and therefore the context coefficient. This study investigates an alternative approach: estimating cluster-level predictor on the latent cluster mean by using multilevel latent. A comparison is made of the accuracy of the context coefficient and standard error under a wide range of conditions. Results reveal that bias for context effect is small in multilevel latent model. Maximum likelihood (ML) estimator yields more accurate standard error estimation than robust maximum likelihood (MLR) when cluster number is small (less than 50). Very small cluster sample sizes (less than 10) should be avoided because they lack power and empirical sampling variance.

Published in American Journal of Theoretical and Applied Statistics (Volume 6, Issue 5)
DOI 10.11648/j.ajtas.20170605.11
Page(s) 221-227
Creative Commons

This is an Open Access article, distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution and reproduction in any medium or format, provided the original work is properly cited.

Copyright

Copyright © The Author(s), 2017. Published by Science Publishing Group

Keywords

Multilevel Latent Model, Context Effect, Parameter Estimate Accuracy, Standard Error, Power

References
[1] Marsh, H. W., Ludtke, O., Robitzsch, A., Trautwein, U., Asparouhov, T., Muthen, B., & Nagengast, B. (2009). Doubly-latent school contextual effects: Integrating multilevel and structural equation approaches to control measurement and sampling error. Multivariate Behavioral Research, 44(6), 764-802.
[2] Raykov, T. (2007). Longitudinal analysis with regressions among random effects: A latent variable modeling approach. Structural Equation Modeling, 14(1), 146-169.
[3] Shin, Y., & Raudenbush, S. W. (2010). A latent cluster-mean approach to the contextual effects model with missing data. Journal of Educational and Behavioral Statistics, 35, 26-53.
[4] Ludtke, O., Marsh, H. W., Robitzsch, A., Trautwein, U., Asparouhov, T., & Muthen, B. (2008). The multilevel latent covariate model: A new, more reliable approach to group-Level effects in contextual studies. Psychological Methods, 13(3), 203-229.
[5] Ludtke, O., Marsh, H. W., Robitzsch, A., & Trautwein, U. (2011). A 2*2 taxonomy of multilevel latent contextual models: Accuracy - bias trade-offs in full and partial error correction models. Psychological Methods, 16(4), 444-467.
[6] Asparouhov, T., & Muthen, B. (2006). Constructing covariates in multilevel regression. Mplus website. Retrieved from http://www.statmodel.com/download/webnotes/webnote11.pdf
[7] Preacher, K. J., Zyphur, M. J., & Zhang, Z. (2010). A general multilevel SEM framework for assessing multilevel mediation. Psychological Methods, 15(3), 209-233.
[8] Lopes, P. N., Mestre, J. M., Guil, R., Kremenitzer, J. P., & Salovey, P. (2012). The Role of Knowledge and Skills for Managing Emotions in Adaptation to School Social Behavior and Misconduct in the Classroom. American Educational Research Journal, 49(4), 710-742.
[9] Hox, J. J., Maas, C. J. M., & Brinkhuis, M. J. S. (2010). The effect of estimation method and sample size in multilevel structural equation modeling. Statistica Neerlandica, 64(2), 157-170.
[10] Maas, C. J. M., & Hox, J. J. (2005). Sufficient sample size for multilevel modeling. Methodology, 1(3), 86-92.
[11] Scherbaum, C. A., & Ferreter, J. M. (2009). Estimating statistical power and required sample sizes for organizational research using multilevel modeling. Organizational Research Methods, 12, 347-367.
[12] Snidjers, T. A. B., & Bosker, R. J. (1999). Multilevel analysis: An introduction to basic and advance multilevel modeling. London: Sage.
[13] Maas, C. J. M., & Hox, J. J. (2004). Robustness issues in multilevel regression analysis. Statistica Neerlandica, 58(2), 127-137.
[14] O’Brien, R. G. (1981). A sample test for variance effects in experimental designs. Psychological Bulletin, 89(3), 570-574.
[15] Zitzmann, S., Lüdtke, O., & Robitzsch, A. (2015). A bayesian approach to more stable estimates of group-level effects in contextual studies. Multivariate Behavioral Research, 50(6), 688-705.
[16] Zitzmann, S., Lüdtke, O., Robitzsch, A., & Marsh, H. W. (2016). A bayesian approach for estimating multilevel latent contextual models. Structural Equation Modeling: A Multidisciplinary Journal, 23(5), 1-20.
Cite This Article
  • APA Style

    Miao Gao. (2017). Estimating the Context Effect in a Multilevel Latent Model with Small Sample Sizes: A Monte Carlo Simulation Study. American Journal of Theoretical and Applied Statistics, 6(5), 221-227. https://doi.org/10.11648/j.ajtas.20170605.11

    Copy | Download

    ACS Style

    Miao Gao. Estimating the Context Effect in a Multilevel Latent Model with Small Sample Sizes: A Monte Carlo Simulation Study. Am. J. Theor. Appl. Stat. 2017, 6(5), 221-227. doi: 10.11648/j.ajtas.20170605.11

    Copy | Download

    AMA Style

    Miao Gao. Estimating the Context Effect in a Multilevel Latent Model with Small Sample Sizes: A Monte Carlo Simulation Study. Am J Theor Appl Stat. 2017;6(5):221-227. doi: 10.11648/j.ajtas.20170605.11

    Copy | Download

  • @article{10.11648/j.ajtas.20170605.11,
      author = {Miao Gao},
      title = {Estimating the Context Effect in a Multilevel Latent Model with Small Sample Sizes: A Monte Carlo Simulation Study},
      journal = {American Journal of Theoretical and Applied Statistics},
      volume = {6},
      number = {5},
      pages = {221-227},
      doi = {10.11648/j.ajtas.20170605.11},
      url = {https://doi.org/10.11648/j.ajtas.20170605.11},
      eprint = {https://article.sciencepublishinggroup.com/pdf/10.11648.j.ajtas.20170605.11},
      abstract = {In multilevel modeling, the relationships between the criterion and predictors are investigated at different levels. Often, the cluster-level predictors are measured by aggregating the individual-level measures. However, the aggregated cluster-level predictors do not always reliably measure the cluster-level regression coefficient, and therefore the context coefficient. This study investigates an alternative approach: estimating cluster-level predictor on the latent cluster mean by using multilevel latent. A comparison is made of the accuracy of the context coefficient and standard error under a wide range of conditions. Results reveal that bias for context effect is small in multilevel latent model. Maximum likelihood (ML) estimator yields more accurate standard error estimation than robust maximum likelihood (MLR) when cluster number is small (less than 50). Very small cluster sample sizes (less than 10) should be avoided because they lack power and empirical sampling variance.},
     year = {2017}
    }
    

    Copy | Download

  • TY  - JOUR
    T1  - Estimating the Context Effect in a Multilevel Latent Model with Small Sample Sizes: A Monte Carlo Simulation Study
    AU  - Miao Gao
    Y1  - 2017/09/04
    PY  - 2017
    N1  - https://doi.org/10.11648/j.ajtas.20170605.11
    DO  - 10.11648/j.ajtas.20170605.11
    T2  - American Journal of Theoretical and Applied Statistics
    JF  - American Journal of Theoretical and Applied Statistics
    JO  - American Journal of Theoretical and Applied Statistics
    SP  - 221
    EP  - 227
    PB  - Science Publishing Group
    SN  - 2326-9006
    UR  - https://doi.org/10.11648/j.ajtas.20170605.11
    AB  - In multilevel modeling, the relationships between the criterion and predictors are investigated at different levels. Often, the cluster-level predictors are measured by aggregating the individual-level measures. However, the aggregated cluster-level predictors do not always reliably measure the cluster-level regression coefficient, and therefore the context coefficient. This study investigates an alternative approach: estimating cluster-level predictor on the latent cluster mean by using multilevel latent. A comparison is made of the accuracy of the context coefficient and standard error under a wide range of conditions. Results reveal that bias for context effect is small in multilevel latent model. Maximum likelihood (ML) estimator yields more accurate standard error estimation than robust maximum likelihood (MLR) when cluster number is small (less than 50). Very small cluster sample sizes (less than 10) should be avoided because they lack power and empirical sampling variance.
    VL  - 6
    IS  - 5
    ER  - 

    Copy | Download

Author Information
  • College of Education, Nanjing Normal University, Nanjing, China

  • Sections