Review Article

Statistical Procedures Used in Pretest-Posttest Control Group Design: A Review of Papers in Five Iranian Journals

Abstract

The pretest-posttest control group design is one of the most widely used quantitative experimental design models for evaluating the efficacy of programs, treatments, and interventions. Despite the prevalence and utility of this research design, best practices for data analytical procedures are not clearly defined. Invalid results decrease the chance of generalization. Given that Iranian Journals are interested in publishing pretest-posttest control group design studies, it is important to denote the accuracy of them. The aim of the current study is to explore the correct procedure for using ANCOVA in pretest-posttest control group designs to mitigate the potential limitations of this approach. This study explores the use of ANCOVA in pretest-posttest control group design. It has been done by analyzing data from experimental studies published in five Iranian journals indexed in PubMed or Scopus between 2011 and 2018. The results indicate that among the 280 published experimental studies in these journals, 53 papers (18.9 percent) used ANCOVA as the statistical test in pretest-posttest studies. The power of the test represents the probability of detecting differences between the groups being compared when such differences exist. Our analysis concludes that ANCOVA, which runs a multiple linear regression, is a suitable method for comparing and examining pretest-posttest study designs. Implications of this study have potential utility for researchers employing the use of pretest-posttest control group designs in various fields in and outside of Iran.

1. Schmidt NA, Brown JM. Evidence-Based Practice for Nurses: Jones & Bartlett Learning, LLC; 2014.
2. Dugard P, Todman J. Analysis of pre‐test‐post‐test control group designs in educational research. Educ Psychol 1995;15:181-98.
3. Johnson B, Christensen L. Educational Research: Quantitative, Qualitative, and Mixed Approaches: SAGE Publications; 2010.
4. Lee J, Lee Y, Kim MH. Effects of Empathy-based Learning in Elementary Social Studies. Asia Pac Educ Res 2018;27:509-21.
5. Zhao W, Berger V. Imbalance control in clinical trial subject randomization—from philosophy to strategy. J Clin Epidemiol 2018;101:116-8.
6. Roberts MC, Ilardi SS. Handbook of Research Methods in Clinical Psychology: Wiley; 2008.
7. Ellis PD. The Essential Guide to Effect Sizes: Statistical Power, Meta-Analysis, and the Interpretation of Research Results: Cambridge University Press; 2010.
8. Vogt WP, Gardner DC, Vogt ER, Haeffele LM. Selecting the Right Analyses for Your Data: Quantitative, Qualitative, and Mixed Methods: Guilford Publications; 2014.
9. Chase LJ, Chase RB. A statistical power analysis of applied psychological research. J App Psychol 1976;61:234-7.
10. Lipsey MW. Design Sensitivity: Statistical Power for Experimental Research: SAGE Publications; 1990.
11. Cohen J. Statistical Power Analysis for the Behavioral Sciences: Taylor & Francis; 2013.
12. Leong FTL, Austin JT. The Psychology Research Handbook: A Guide for Graduate Students and Research Assistants: SAGE Publications; 2006.
13. Mayers A. Introduction to Statistics and SPSS in Psychology: Pearson Education Limited; 2013.
14. Haghdoost A. Do You Want to Gain a Profound Insight into Sample Size and Statistical Power. Iran J Epidemiol 2009;5:57-63.
15. Rosnow RL, Rosenthal R. Definition and interpretation of interaction effects. Psychol Bull 1989;105:143-6.
16. Sullivan GM, Feinn R. Using effect size—or why the P value is not enough. J Grad Med Educ 2012;4:279-82.
17. Adams KA, Lawrence EK. Research Methods, Statistics, and Applications: SAGE Publications; 2014.
18. Lavrakas PJ. Encyclopedia of Survey Research Methods: A-M: SAGE Publications; 2008.
19. Cohen L, Manion L, Morrison K. Research Methods in Education: Taylor & Francis; 2007.
20. Polit DF, Beck CT. Nursing research: Generating and assessing evidence for nursing practice: Lippincott Williams & Wilkins; 2017.
21. Dimitrov DM, Rumrill Jr PD. Pretest-posttest designs and measurement of change. Work 2003;20:159-65.
22. Grove SK, Burns N, Gray J. Understanding nursing research: Building an evidence-based practice: Elsevier Health Sciences; 2014.
23. Bonate PL. Analysis of pretest-posttest designs: CRC Press; 2000.
24. Verma J. Data analysis in management with SPSS software: Springer Science & Business Media; 2012.
25. Lin MH. Effects of classroom blogging on ESL student writers: An Empirical reassessment. Asia Pac Educ Res 2014;23:577-90.
26. Reising DL, Carr DE, Tieman S, Feather R, Ozdogan Z. Psychometric testing of a simulation rubric for measuring interprofessional communication. Nurs Educ Perspect 2015;36:311-6.
27. Tabachnick BG, Fidell LS, Ullman JB. Using Multivariate Statistics: Pearson; 2018.
28. Pace L. The Excel Data and Statistics Cookbook, Third Edition: Lulu.com; 2013.
29. Jackson S. Research Methods: A Modular Approach: Cengage Learning; 2007.
30. Kirk RE. Experimental Design: Procedures for the Behavioral Sciences: SAGE Publications; 2012.
31. Sapp M. Basic Psychological Measurement, Research Designs, and Statistics Without Math: Charles C. Thomas Publisher; 2006.
32. Miller GA, Chapman JP. Misunderstanding analysis of covariance. J Abnorm Psychol 2001;110:40-8.
33. Field A. Discovering Statistics using IBM SPSS Statistics 4e + eBook + WebAssign Single Term + SPSS Version 23.0. 5th ed. London: SAGE Publications; 2018.
34. Leech NL, Barrett KC, Morgan GA. IBM SPSS for Intermediate Statistics: Use and Interpretation, Fifth Edition: Taylor & Francis; 2014.
35. Gliner JA, Morgan GA, Harmon RJ. Pretest-posttest comparison group designs: Analysis and interpretation. J Am Acad Child Adolesc Psychiatry 2003;42:500-3.
36. Lee JH. Experimental methodology in English teaching and learning: Method features, validity issues, and embedded experimental design. Engl Teach Pract Crit 2012;11:25-43.
37. Vickers AJ, Altman DG. Statistics notes: analysing controlled trials with baseline and follow up measurements. Br Med J 2001;323:1123-4.
38. Bonita R, Beaglehole R, Kjellström T, santé Omdl, Organization WH. Basic Epidemiology: World Health Organization; 2006.
Files
IssueVol 61 No 10 (2023) QRcode
SectionReview Article(s)
DOI https://doi.org/10.18502/acta.v61i10.15657
Keywords
Pretest-posttest study Analysis of covariance Nursing
Rights and permissions
Creative Commons License This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
How to Cite
1.
Dehghan Nayeri N, Alazmani Noodeh F, Sharif Nia H, Yaghoobzadeh A, Allen KA, Goudarzian AH. Statistical Procedures Used in Pretest-Posttest Control Group Design: A Review of Papers in Five Iranian Journals. Acta Med Iran. 2024;61(10):584-591.