AN ANALYSIS OF THE LONGITUDINAL MEASUREMENT INVARIANCE OF THE SOCIAL PRESENCE SCALE DEVELOPED FOR OPEN AND DISTANCE LEARNING ENVIRONMENTS

Year 2020, Volume: 21 Issue: Special Issue-IODL, 97 - 110, 17.07.2020

Murat Dogan Sahın , Hakan Kılınc , Hakan Altınpulluk

https://doi.org/10.17718/tojde.770937

Cited By: 1

Abstract

As with all fields of social sciences, open and distance learning studies frequently use self-reports in their collection of data. Similarly, as with all measurement tools, proof of construct validity through statistical processes is important for testing hypotheses based on the findings, making decisions, and essentially ensuring the tools and methods used are fit for purpose. During construct validity testing processes, the reporting of findings regarding measurement invariance of a scale is an important element that is often neglected. This study aims to portray the longitudinal invariance of the Social Presence Scale (Cakmak, Cebi & Kan, 2014), which is frequently used to determine the social presence in open and distance learning environments research and has confirmed construct validity in the literature, through repeated measurements obtained in an experimental study. This research was conducted on 280 learners in a 3-month interval. The data gathered was analyzed for measurement invariance using the Mplus 7.0 software package in accordance with the four stages described in the literature. The measurement invariance tested in each stage were studied for ΔCFI, and ΔRMSEA values in addition to likelihood chi-square. The findings indicate the conditions required for measurement invariance in each stage, or in other words, the longitudinal invariance of the scale was achieved. The findings of this study may provide a precedent for similar studies in the future.

Keywords

Measurement invariance, longitudinal data, confirmatory factor analysis, open and distance learning, social presence

References

• Akcaoglu, M., & Lee, E. (2016). Increasing social presence in online learning through small group discussions. The International Review of Research in Open and Distributed Learning, 17(3). Doi: 10.19173/irrodl.v17i3.2293
• Annand, D. (2011). Social presence within the community of inquiry framework. The International Review of Research in Open and Distributed Learning, 12(5), 40-56. Doi: 10.19173/irrodl.v12i5.924
• Basusta, N.B.U. (2010). Olcme esdegerligi. Egitimde ve Psikolojide Olcme ve Degerlendirme Dergisi, 1(2), 58-64.
• Bentler, P.M., & Bonnet, D.C. (1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88(3), 588-606.
• Borup, J., West, R. E., & Graham, C. R. (2012). Improving online social presence through asynchronous video. The Internet and Higher Education, 15(3), 195–203. Doi: 10.1016/j.iheduc.2011.11.001
• Boston, W., Diaz, S. R., Gibson, A. M., Ice, P., Richardson, J., & Swan, K. (2009). An exploration of the relationship between indicators of the community of inquiry framework and retention in online programs. Journal of Asynchronous Learning Networks, 13(3).
• Cakmak, E. K., Cebi, A., & Kan, A. (2014). E-ogrenme ortamlarina yonelik “sosyal bulunusluk olcegi” gelistirme calismasi. Kuram ve Uygulamada Egitim Bilimleri, 14(2), 755-768. Doi: 10.12738/estp.2014.2.1847
• Chen F.F., Sousa K.H. & West SG. (2005). Testing measurement invariance of second-order factor models. Structural Equation Modeling, 14, 471–492. Doi: 10.1207/s15328007sem1203_7
• Cheung, G. W., & Rensvold, R. B. (2002) Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9(2), 233-255. Doi: 10.1207/S15328007SEM0902_5
• Cheung, G.W., & Rensvold R.B. (1999). Testing factorial invariance across groups: A reconceptualization and proposed new method. Journal of Management, 25, 1–27.
• Cobb, S. C. (2009). Social presence and online learning: A current view from a research perspective. Journal of Interactive Online Learning, 8(3), 241–254.
• Dimitrov, D. D. (2010). Testing for factorial invariance in the context of construct validation. Measurement and Evaluation in Counseling and Development, 43(2). 121-149. Doi: 10.1177/0748175610373459
• Elcicek, M., Erdemci, H., & Karal, H. (2018). Examining the relationship between the levels of digital citizenship and social presence for the graduate students having online education. Turkish Online Journal of Distance Education, 19(1), 203-214. Doi: 10.17718/tojde.382801
• Ercan, Y., & Bulbul, T. (2019). Uzaktan egitim ortamlarinda iletisimci bicimlerinin sosyal buradaliga etkisi. Trakya University Journal of Social Science, 21(2). Doi: 10.26468/trakyasobed.630333
• Ferrer E., Balluerka N., & Widaman K.F. (2008). Factorial invariance and the specification of second-order latent growth models. Methodology, 4, 22–36. Doi: 10.1027/1614-2241.4.1.22
• Garrison, D. R. (2011). E-learning in the 21st century: A framework for research and practice. Routledge.
• Garrison, D. R., & Akyol, Z. (2013). The community of inquiry theoretical framework. G. M. Moore (Ed.), Handbook of distance education (pp. 104–120). New York, NY: Routledge.
• Hooper, D., Coughlan, J., & Mullen, M.R. (2008). Structural equation modelling: Guidelines for determining model fit. Electronic Journal of Business Research Methods, 6(1), 53-60.
• Horzum, M. B. (2015). Interaction, structure, social presence, and satisfaction in online learning. Eurasia Journal of Mathematics, Science & Technology Education, 11(3). Doi: 10.12973/eurasia.2014.1324a
• Hostetter, C. (2013). Community matters: Social presence and learning outcomes. Journal of the Scholarship of Teaching and Learning, 13(1), 77-86.
• Hostetter, C., & Busch, M. (2006). Measuring up online: The relationship between social presence and student learning satisfaction. Journal of the Scholarship of Teaching and Learning, 1-12.
• Hu, L.T., & Bentler, P.M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1-55. Doi: 10.1080/10705519909540118
• Joksimovic, S., Gasevic, D., Kovanovic, V., Riecke, B. E., & Hatala, M. (2015). Social presence in online discussions as a process predictor of academic performance. Journal of Computer Assisted Learning, 31(6), 638-654. Doi: 10.1111/jcal.12107
• Jorge, I. (2010). Social presence and cognitive presence in an online training program for teachers of Portuguese: Relation and methodological issues. IODL and ICEM 2010 Joint Conference and Media Days, (pp.427–435).
• Kang, M., & Im, T. (2013). Factors of learner–instructor interaction which predict perceived learning outcomes in online learning environment. Journal of Computer Assisted Learning, 29(3), 292-301. Doi: 10.1111/jcal.12005
• Kilinc, H. (2020). Cevrimici grup tartismalarinin ogrenenler uzerindeki etkisinin cesitli degiskenler acisindan incelenmesi: Anadolu Universitesi Acikogretim Fakultesi ornegi. Unpublished doctoral thesis. Anadolu University, Institute of Social Science, Eskisehir.
• Kreijns, K., Van Acker, F., Vermeulen, M., & Van Buuren, H. (2014). Community of inquiry: Social presence revisited. E-learning and Digital Media, 11(1), 5-18. Doi: 10.2304/elea.2014.11.1.5
• Little T.D. (1997). Mean and covariance structures (MACS) analyses of cross-cultural data: Practical and theoretical issues. Multivariate Behavioral Research, 32, 53–76. Doi: 10.1207/s15327906mbr3201_3
• Lowenthal, P. R. (2010). The evolution and influence of social presence theory on online learning. T. T. Kidd (Ed.), Online Education and Adult Learning: New Frontiers for Teaching Practices. Hershey, PA: IGI Global.
• Marsh, A.J.S., Parker, P.D., & Morin, A.J.S. (2015). Invariance testing across samples and time: Cohort sequence analysis of perceived body composition. In N. Ntoumanis, & N. Myers (Eds.), An Introduction to Intermediate and Advanced Statistical Analyses for Sport and Exercise Scientists. Wiley.
• Matanaghi, A. (2015). Online learning readiness level and perceived social presence of the teacher candidate’s in the online learning environment an EMU Example. (Master’s thesis, Eastern Mediterranean University (EMU)-Dogu Akdeniz Universitesi (DAU)).
• Millsap, R.E. & Meredith, W. (2007). Factorial invariance: Historical perspectives and new problems. Factor Analysis At 100: Historical Developments and New Directions. Mahwah, NJ: Erlbaum.
• Muthen, L. K., & Muthen, B. O. (1998-2020). Mplus User’s Guide. Sixth Edition. Los Angeles, CA: Muthen & Muthen.
• Nesselroade, J.R. (1983). Temporal selection and factor invariance in the study of development and change. Life-Span Development and Behavior. New York: Academic Press.
• Oztok, M., & Brett, C. (2011). Social presence and online learning: A review of research. International Journal of E-Learning & Distance Education, 25(3).
• Putnick D. L., & Bornstein, M. H. (2016). Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Developmental Review, 41, 71-90. Doi: 10.1016/j.dr.2016.06.004
• Rensvold R.B., & Cheung G.W. (1998). Testing measurement model for factorial invariance: A systematic approach. Educational and Psychological Measurement, 58, 1017–1034. Doi: 10.1177/0013164498058006010
• Richardson, J., & Swan, K. (2003). Examing social presence in online courses in relation to students’ perceived learning and satisfaction. Journal of Asynchronus Learning Networks. 7(1), 68-88.
• Russo, T. C., & Benson, S. (2005). Learning with invisible others: Perceptions of online presence and their relationship to cognitive and affective learning. International Forum of Educational Technology and Society. 8(1), 54-62.
• Strong, R., Irby, T. L., Wynn, J. T., & McClure, M. M. (2012). Investigating students’ satisfaction with elearning courses: the effect of learning environment and social presence. Journal of Agricultural Education, 53(3), 98-110. Doi: 10.5032/jae.2012.03098
• Tabachnick, B.G., & Fidell, L.S. (2008). Using multivariate statistics. New York: Allyn and Bacon.
• Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4–70.
• Whiteside, A. L. (2015). Introducing the social presence model to explore online and blended learning experiences. Online Learning, 19(2).
• Widaman, K. F., & Reise, S. P. (1997). Exploring the measurement invariance of psychological instruments: Applications in the substance use domain. In K. J. Bryant, M. Windle, & S. G. West (Eds.), The science of prevention: Methodological advances from alcohol and substance abuse research (pp. 281-324). Washington, DC, US: American Psychological Association.
• Widaman, K. F., Ferrer, E., & Conger, R. D. (2010). Factorial invariance within longitudinal structural equation models: Measuring the same construct across time. Child Development Perspectives, 4(1), 10–18. Doi: 10.1111/j.1750-8606.2009.00110.x