Assessment of an Adolescent Digital Citizenship Scale: Examining Dimensionality, Measurement Invariance and External Validity
DOI:
https://doi.org/10.30827/relieve.v30i1.25367Keywords:
digital citizenship, adolescence, measurement, validity, reliabilityAbstract
The positive online behavior effects of digital citizenship have increasingly attracted the attention of scholars. This study designed and tested the psychometric properties of an Adolescent Digital Citizenship Scale (DCS-A) in two independent samples of Mexican secondary students (Sample 1, M age = 13.2 years, SD = 1.5 and Sample 2, M age = 13.4 years, SD = 1.4; N1 = 750, N2 = 750). We examined content, factorial, discriminant, concurrent validity, and reliability. We also tested the cross-sample and gender invariance. Confirmatory factor analysis (CFA) demonstrated goodness-of-fit on a second-order factorial model that displays three first-order factors (online ethic, online civic engagement, and online diversity acceptance). Cross-validation confirmed the factorial structure stability of the DCS-A across the independent sample. The result demonstrated the equivalence of the measurement model in both genders (configural, metric, and scalar invariance). The latent means comparison indicates that females held greater online ethics, online civic engagement, and online inclusive behaviors than males. Finally, the concurrent validity of the scale was supported by finding a positive relationship between DCS-A dimensions and defender behavior and a negative association with passive and reinforces interventions in cyberbullying events. These results suggest that the DCS-A is a theoretically and psychometrically grounded measure of digital citizenship in adolescents.
Downloads
References
Almerich, G., Suárez-Rodríguez, J., Díaz-García, I., & Orellana, N. (2021). The influence of using ICT in high-skills competences and ICT competences. A structural model. Education and Information Technologies, 26, 3845–3869. https://doi.org/10.1007/s10639-021-10437- y
Al-Zahrani, A. (2015). Toward digital citizenship: Examining factors affecting participation and involvement in the Internet society among higher education students. International Education Studies, 8(12), 203–217. https://doi.org/10.5539/ies. v8n12p203
Anderson, E. L., Steen, E., & Stavropoulos, V. (2017). Internet use and problematic Internet use: a systematic review of longitudinal research trends in adolescence and emergent adulthood. International Journal of Adolescent and Youth, 22(4), 430–454. https://doi.org/10.1080/02673843 .2016.1227716
Areepattannil, S., & Khine, M. S. (2017). Early adolescents’ use of information and communication technologies (ICTs) for social communication in 20 countries: Examining the roles of ICT-related behavioral and motivational characteristics. Computers in Human Behavior, 73, 263–272. https://doi.org/10.1016/j.chb.2017.03. 058
Balakrishnan, V. (2018). Actions, emotional reactions and cyberbullying – From the lens of bullies, victims, bully-victims and bystanders among Malaysian young adults. Telematics and Informatics, 35(5), 1190–1200. https://doi.org/10.1016/j.tele.2018.02 .002
Bandalos, D. L., & Finney, S. J. (2019). Factor analysis. Exploratory and confirmatory. In G. R. Hancock, L. M. Stapleton, & R. O. Mueller (Eds.), The review guide to quantitative methods in the social sciences (2nd ed., pp. 98–122). Routledge.
Bauman, S., Yoon, J., Iurino, C., & Hackett, L. (2020). Experiences of adolescent witnesses to peer victimization: The bystander effect. Journal of School Psychology, 80, 1–14. https://doi.org/10.10 16/j.jsp.2020.03.002
Bauml, M., Smith, V. D., & Blevins, B. (2022). “Who cares?”: Young adolescents’ perceived barriers to civic action. Research in Middle Level Education, 45(3), 1–20. https://doi.org/10.1080/19404476.2022.20 33069
Brown, T. A. (2015). Confirmatory factor analysis for applied research (2nd ed.). The Guilford Press.
Byrne, B. M. (2012). Structural equation modeling with Mplus. Basic concepts, applications, and programming. Routledge.
Castaño-Pugarín, S. A., Suárez-Betancur, N., Tilano Vega, L. M., Herrera López, H. M. (2021). Internet, social media and online hate speech. Systematic review. Aggression and Violent Behavior, 58, Artículo e101608. https://doi.org/10.1016/j.avb.202 1.101608
Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9(2), 233–255. https://doi.org/10. 1207/S15328007SEM0902_5
Choi, M. (2016). A concept analysis of digital citizenship for democratic citizenship education in the internet age. Theory & Research in Social Education, 44(4), 565–607. https://doi.org/10.1080/00933104.201 6.1210549
Choi, M., Glassman, M., & Cristol, D. (2017). What it means to be a citizen in the internet age: Development of a reliable and valid digital citizenship scale. Computers & Education, 107, 100–112. https://doi.org/ 10.1016/j.compedu.2017.01.002
Claravall, E. B., & Evans-Amalu, K. (2020). Self-regulation, and compassion: A critical triad to develop anti-racist digital citizenship in the time of pandemic. Journal of International Social Studies, 10(2), 44–59. https://files.eric.ed.gov/fulltext/EJ1266 427.pdf
Coyne, S. M., Padilla-Walker, L. M., Fraser, A. M., Fellows, K., & Day, R. D. (2014). “Media time = family time”: Positive media use in families with adolescents. Journal of Adolescent Research, 29(5), 663–688. https://doi.org/10.1177/0743558414538316
Crocetti, E., Erentaitė, R., & Žukauskienė, R. (2014). Identity styles, positive youth development, and civic engagement in adolescence. Journal of Youth and Adolescence, 43, 1818–1828. https://doi.or g/10.1007/s10964-014-0100-4
Curran, M. B. X., & Ribble, M. (2017). P-20 Model of digital citizenship. New Directions for Student Leadership, 2017(153), 35–46. https://doi.org/10.1002/ yd.20228
Dedebali, N. C., & Dasdemir, I. (2019). Social studies teacher candidates’ perception of digital citizenship. International Journal of Educational Methodology, 5(3), 465–477. https://doi.org/10.12973/ijem.5.3.465
DeSmet, A., De Bourdeaudhuij, I., Walrave, M., & Vandebosch, H. (2019). Associations between bystander reactions to cyberbullying and victims’ emotional experiences and mental health. Cyberpsychology, Behavior, and Social Networking, 22(10), 648–656. https://doi.or g/10.1089/cyber.2019.0031
Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39–50. https://doi.org/10.1177/0022 24378101800104
Funder, D. C., & Ozer, D. J. (2019). Evaluating effect size in psychological research: Sense and nonsense. Advances in Methods and Practices in Psychological Science, 2(2), 156–168. https://doi.org/10.1177/25152459 19847202
Garaigordobil, M., & Larrain, E. (2020). Bullying and cyberbullying in LGBT adolescents: Prevalence and effects on mental health. Comunicar, 28(62), 77–87. https://doi.org/10.3916/C62-2020-07
Guasp-Coll, M., Navarro-Mateu, D., Lacomba-Trejo, L., Giménez-Espert, M. C., & Prado-Gascó, V. J. (2021). Emotional skills in adolescents’ attitudes towards diversity: Regression models vs qualitative comparative analysis models. Current Psychology, Advanced online. https://doi.o rg/10.1007/s12144-020-01319-6
Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2010). Multivariate data analysis (7th ed.). Pearson.
Hair, J. F., Hult, G. T. M., Ringle, C. M., & Sarstedt, M. (2017). A primer on partial least squares structural equation modeling (2nd ed.). Sage.
Harrison, T., & Polizzi, G. (2022). (In) civility and adolescents’ moral decision making online: drawing on moral theory to advance digital citizenship education. Education and Information Technologies, 27, 3277–3297. https://doi.org/10.1007/s10639-021-10710-0
Heath, M. K. (2018). What kind of (digital) citizenship? A between-studies analysis of research and teaching for democracy. International Journal of Information and Learning Technology, 35(5), 342–356. https://doi.org/10.1108/IJILT-06-2018-006 7
Hernández-Martín, A., Martín-del-Pozo, M., & Iglesias-Rodríguez, A. (2021). Pre-
adolescents’ digital competences in the area of safety. Does frequency of social media use mean safer and more knowledgeable digital usage? Education and Information Technologies, 26(1), 1043–1067. https:// doi.org/10.1007/s10639-020-10302-4
Hinduja, S., & Patchin, J. W. (2008). Cyberbullying: An exploratory analysis of factors related to offending and victimization. Deviant Behavior, 29(2), 129–156. https://doi.org/10.1080/01639620 701457816
Ho, R. (2006). Handbook of univariate and multivariate data analysis and interpretation with SPSS. Chapman & Hall/CRC.
Holfeld, B. (2014). Perceptions and attributions of bystanders to cyber bullying. Computers in Human Behavior, 38, 1–7. https://doi.org/10.1016/j.chb.2014.05.012
Instituto Nacional para la Evaluación de la Educación (2019). Panorama educativo de México. Indicadores del Sistema Educativo Nacional 2018 Educación básica y media, superior. https://www.inee.edu.mx/wp-con tent/uploads/2019/08/P1B117.pdf
International Telecommunication Union (2021). New data from ITU suggest ‘COVID-19 connectivity boost’- world’s poorest being left far behind. https:// www.itu.int/en/mediacentre/Pages/PR-202 1-11-29-FactsFigures.aspx
Isman, A., & Gungoren, O. C. (2014). Digital citizenship. The Turkish Online Journal of Educational Technology, 13(1), 73–77. https://eric.ed.gov/?id=EJ1018088
Jones, L. M., & Mitchell, K. J. (2016). Defining and measuring youth digital citizenship. New Media & Society, 18(9), 2063–2079. https://doi.org/10.1177/146144 48155777 97
Jugert, P., Eckstein, K., Noack, P., Kuhn, A., & Benbow, A. (2013). Offline and online civic engagement among adolescents and young adults from three ethnic groups. Journal of Youth and Adolescence, 42, 123–135. https://doi.org/10.1007/s10964-012-9 805-4
Kara, N. (2018). Understanding university students’ thoughts and practices about digital citizenship: A mixed methods study. Educational Technology & Society, 21(1), 172–185. https://www.jstor.org/stable/10.2 307/26273878
Kim, M., & Choi, D. (2018). Development of Youth Digital Citizenship Scale and implications for educational setting. Educational Technology & Society, 21(1), 155–171. https://www.jstor.org/stable/10.2 307/26273877
Kim, S., & Han, Y. (2020). Different trajectories of citizenship across latent classes of adolescent online risk behavior. Children and Youth Services Review, 109, Article e104731. https://doi.org/10.1016/j. childyouth.2019.104731
Lambe, L. J., Cioppa, V. D., Hong, I. K., & Craig, W. H. (2019). Standing up to bullying: A social ecological review of peer defending in offline and online contexts. Aggression and Violent Behavior, 45, 51–74. https://doi.org/10.1016/j.avb.2018.05.0 07
Lau, W. W. F., & Yuen, A. H. K. (2014). Developing and validating of a perceived ICT literacy scale for junior secondary school students: Pedagogical and educational contributions. Computers & Education, 78, 1–9. https://doi.org/10.1016/ j.compedu.2014.04.016
Lee, C.-H., Moak, S., & Walker, J. T. (2016). Effects of self-control, social control, and social learning on sexting behavior among South Korean Youths. Youth & Society, 48(2), 242–264. https://doi.org/10.1177%2 F0044118X13490762
Lyons, R. (2012). Investigating student gender and grade level differences in digital citizenship behavior (Publication No. 1038378905) [Doctoral thesis, Walden University]. ProQuest Dissertation and Theses Global.
Machackova, H., Dedkova, L., Sevcikova, A., & Cerna, A. (2018). Bystanders’ supportive and passive responses to cyberaggression. Journal of School Violence, 17(1), 99–110. https://doi.org/10. 1080/15388220.2016.1222499
Marcum, C. D., Higgins, G. E., Ricketts, M. L., & Wolfe, S. E. (2014). Hacking in high school: Cybercrime perpetration by juveniles. Deviant Behavior, 35(7), 581–591. https://doi.org/10.1080/01639625.201 3.867721
Martin, F., Hunt, B., Wang, C., & Brooks, E. (2020). Middle school student perception of technology use and digital citizenship practices. Computers in the Schools, 37(3), 196–215. https://doi.org/10.1080/07380569 .2020.1795500
Mueller, R. O., & Hancock, G. R. (2019). Structural equation modeling. In G. R. Hancock, L. M. Stapleton, & R. O. Mueller (Eds.), The reviewer’s guide to quantitative methods in the social sciences (2nd ed., pp. 445–456). Routledge.
Mueller, M. K., Phelps, E., Bowers, E. P., Agans, J. P., Urban, J. B., & Lerner, R. M. (2011). Youth development program participation and intentional self-regulation skills: Contextual and individual bases of pathways to positive youth development. Journal of Adolescence, 34(6), 1115–1125. https://doi.org/10.1016/j.adolescence.2011.07.010
Muthén, L. K., & Muthén, B. O. (2017). Mplus user’s guide (8th ed.). Muthén & Muthén.
Nordin, M. S., Ahmad, T. B. T., Zubairi, A. M., Ismail, N. A. H., Rahman, A. H. A., Trayek, F. A. A., & Ibrahim, M. B. (2016). Psychometric properties of a Digital Citizenship Questionnaire. International Education Studies, 9(3), 71–80. https://doi. org/10.5539/ies.v9n3p71
Notten, N., & Nikken, P. (2016). Boys and girls taking risk online: A gendered perspective on social context and adolescents’ risky online behavior. New Media & Society, 18(6), 966–988. https://doi.org/10.1177%2F1461444814552379
Organization for Economic Cooperation and Development (2020). Internet access (indicator). https://doi.org/10.1787/69c2b9 97-en
Patterson, L. J., Allan, A., & Cross, D. (2017). Adolescent bystander behavior in the school and online environments and the implications for interventions targeting cyberbullying. Journal of School Violence, 16(4), 361–375. https://doi.org/10.1080/15 388220.2016.1143835
Peterson, R. A., & Kim, Y. (2013). On the relationship between coefficient alpha and composite reliability. Journal of Applied Psychology, 98(1), 194–198. https://doi.org /10.1037/a0030767
Polit, D. F., Beck, C. T., & Owen, S. V. (2007). Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Research in Nursing & Health, 30(4), 459–467. https://doi.org/10. 1002/nur.20199
Putnick, D. L., & Bornstein, M. H. (2016). Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Developmental Review, 41, 71–90. https://doi.org/10.1016/j.dr.2016.06.004
Reiner, I., Tibubos, A. N., Hardt, J., Müller, K., Wölfling, K., & Beutel, M. E. (2017). Peer attachment, specific patterns of internet use and problematic internet use in male and female adolescents. European Child & Adolescent Psychiatry, 26, 1257–1268. https://doi.org/10.1007/s00787-017-0984-0
Ribble, M. S. (2015). Digital citizenship in schools (3rd ed.). International Society of Technology in Education.
Sarmiento, A., Herrera-López, M., & Zych, I. (2019). Is cyberbullying a group process? Online and offline bystanders of cyberbullying act as defenders, reinforces and outsiders. Computer in Human Behavior, 99, 328–334. https://doi.org/10. 1016/j.chb.2019.05.037
Sherrod, L. R., Flanagan, C., & Youniss, J. (2002). Dimensions of citizenship and opportunities for youth development: The what, why, when, where, and who of citizenship development. Applied Developmental Science, 6(4), 264–272. https://doi.org/10.1207/S1532480xads0604_14
Subramanyam, K., & Smahel, D. (2011). Digital youth: The role of media in development. Springer. https://doi.org/ 10.1007/978-1-4419-6278-2
Tokunaga, R. S. (2010). Following you home from school: A critical review and synthesis of research on cyberbullying victimization. Computers in Human Behavior, 26(3), 277–287. https://doi.or g/10.1016/j.chb.2009.11.014
Torgal, C., Espegale, D. L., Polanin, J. R., Ingram, K. M., Robinson, L. E., El Sheikh, A. J., & Valido, A. (2021). A meta-analysis of school-based cyberbullying prevention programs’ impact on cyber-bystander behavior. School Psychology Review, 52(2), 95–109. http://doi.org/10.1080/2372966X.2 021.1913037
United Nations Educational, Scientific and Cultural Organization (2020). Digital citizenship as a public policy in education in Latin America. https://unesdoc.unesco. org/ark:/482223/pf0000376935_eng
Vlaanderen, A., Bevelander, K. E., & Kleemans, M. (2020). Empowering digital citizenship: An anti-cyberbullying intervention to increase children’s intentions to intervene on behalf of the victim. Computers in Human Behavior, 112, Artículo e106459. https://doi.org/1011 0.1016/ j.chb.2020.106459
Wachs, S., Wright, M. F., Sittichai, R., Singh, R., Biswal, R., Kim, E-M., Yang, S., Gámez-Guadix, M., Almendros, C., Flora, K., Daskalou, V., & Maziridou, E. (2019). Associations between witnessing and perpetrating online hate in eight countries: The buffering effects of problem-focused coping. International Journal of Environmental Research and Public Health, 16(20), Artículo e3992. https://doi. org/10.3390/ijerph16203992
Wang, H., Geng, J., Liu, K., Wei, X., Wang, J., & Lei, L. (2021). Future time perspective and self-control mediate the links between parental autonomy support and adolescents’ digital citizenship behavior. Youth & Society, 54(6), 1077–1096. https://doi.org/ 10.1177/0044118X211020778
Wynd, C. A., Schmidt, B., & Schaefer, M. A. (2003). Two quantitative approaches for estimating content validity. Western Journal of Nursing Research, 25(5), 508–518. https://doi.org/10.1177/01939459032 52998
Zhong, J., Zheng, Y., Huang, X., Mo, D., Gong, J., Li, M., & Huang, J. (2021). Study of the influencing factors of cyberbullying among Chinese college students incorporated with digital citizenship: From the perspective of individual students. Frontiers in Psychology, 12, Artículo e621418. https://doi.org/10.3389/fpsyg.202 1.621418
Zhu, C., Huang, S., Evans, R., & Zhang, W. (2021). Cyberbullying among adolescents and children: A comprehensive review of the global situation, risk factors, and preventive measure. Frontiers in Public Health, 9, Artículo e634909. https://doi.org/ 10.3389/fpubh.2021.63 4909
Zych, I., Farrington, D. P., & Ttofi, M. M. (2019). Protective factors against bullying and cyberbullying: A systematic review of meta-analyses. Aggression and Violent Behavior, 45, 4–19. https://doi.org/10.1016 /j.avb.2018.06.008
![](https://revistaseug.ugr.es/public/journals/41/submission_25367_20918_coverImage_es_ES.png)
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 RELIEVE – Electronic Journal of Educational Research and Evaluation
![Creative Commons License](http://i.creativecommons.org/l/by-nc/4.0/88x31.png)
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
The authors grant non-exclusive rights of exploitation of works published to RELIEVE and consent to be distributed under the Creative Commons Attribution-Noncommercial Use 4.0 International License (CC-BY-NC 4.0), which allows third parties to use the published material whenever the authorship of the work and the source of publication is mentioned, and it is used for non-commercial purposes.
The authors can reach other additional and independent contractual agreements, for the non-exclusive distribution of the version of the work published in this journal (for example, by including it in an institutional repository or publishing it in a book), as long as it is clearly stated that the Original source of publication is this magazine.
Authors are encouraged to disseminate their work after it has been published, through the internet (for example, in institutional archives online or on its website) which can generate interesting exchanges and increase work appointments.
The fact of sending your paper to RELIEVE implies that you accept these conditions.