Abstract
The purpose of this article was to share a standards-aligned instrument designed for evaluating teachers’ educational technology competencies. A survey derived from multiple sets of popular educational technology standards was developed and administered in four phases to a total of 184 teachers. Constructive response, cognitive interviews, and Rasch empirical analyses were used to refine the instrument. These processes resulted in an instrument with evidence to indicate validity and reliability that aligned to the International Society for Technology in Education (ISTE) Standards for Educators, Association for Educational Communications and Technology (AECT) Standards, Association for the Advancement of Computing Education (AACE) Teacher Educator Technology Competencies, and Virtual Learning Leadership (VLL), Quality Matters (QM), and the Digital Learning Collaborative (DLC) National Standards for Quality Online Teaching. This study provides an instrument that may be used by educator preparation providers, school districts, accreditation bodies, or other educational institutions to measure educators’ edtech competencies derived from professional standards.
Similar content being viewed by others
References
AECT (Association for Educational Communications and Technology) (1989). Guidelines for the accreditation of programs in educational communications and information technologies. Washington, DC.
AECT (Association for Educational Communications and Technology) (2012). AECT Standards. Retrieved from https://www.aect.org/docs/AECTstandards2012.pdf
Ayad, F. I., & Ajrami, S. J. (2017). The degree of implementing ISTE standards in technical education colleges of Palestine. The Turkish Journal of Educational Technology, 16(2), 107–118.
Bakir, N. (2016). Technology and teacher education: A brief glimpse of the research and practice that have shaped the field. TechTrends, 60, 21–29. https://doi.org/10.1007/s11528-015-0013-4
Bond, T. G., & Fox, C. M. (2007). Applying the Rasch Model: fundamental measurement in the human sciences (2nd ed.). Lawrence Erlbaum Associates.
Bradley, K. D., & Bradley, J. W. (2006). Challenging the validity of higher education course evaluations. Journal of College Teaching & Learning, 3(8), 63–76.
Bush, B., & Hurlbut, A. (2020). Measuring digital literacy: Instrument development to assess technology usage and integration for continuous program improvement. In D. Schmidt-Crawford (Ed.), Proceedings of Society for Information Technology & Teacher Education International Conference (pp. 981–985). Online: Association for the Advancement of Computing in Education (AACE). https://www.learntechlib.org/primary/p/215849/
Council for the Accreditation of Educator Preparation (2022). 2022 Initial Level Standards. https://caepnet.org/~/media/Files/caep/standards/2022-initial-standards-1-pager-final.pdf?la=en
Chai, C. S., Ling Koh, H., & Teo, Y. H. (2019). Enhancing and modeling teachers’ design beliefs and efficacy of technological pedagogical content knowledge for 21st century quality learning. Journal of Educational Computing Research, 57(2), 360–384.
Cherner, T., Fegely, A. & Gleasman, C. L. (2021). We need to talk about the ISTE Students standards: A critical analysis. In D.C. Gibson & M.N. Ochoa (Eds.), Research Highlights in Technology and Teacher Education (pp. 85–92). AACE. https://www.learntechlib.org/p/220334/
Douglas, J. A. (2011). Competency for quality online teaching. [Doctoral dissertation, University of Virginia]. ProQuest Dissertations & Theses. https://www.proquest.com/docview/875240837
Faison, C. (1994). Integrating technology into teacher education: a modular approach. In J. Willis, B. Robin, & D. Willis (Eds.), Technology and Teacher Education Annual, 1994 (pp. 169–171). Association for the Advancement of Computing in Education.
Foulger, T. S., Buss, R. R., Wetzel, K., & Lindsey, L. (2015). Instructors’ growth in TPACK: teaching technology-infused methods courses to preservice teachers. Journal of Digital Learning in Teacher Education, 31(4), 134–147.
Foulger, T. S., Graziano, K. J., Schmidt-Crawford, D., & Slykhuis, D. A. (2017). Teacher educator technology competencies. Journal of Technology and Teacher Education, 25(4), 413–448. Society for Information Technology & Teacher Education. Retrieved from https://www.learntechlib.org/p/181966/
Foulger, T. S., Graziano, K. J., Schmidt-Crawford, D. A., & Slykhuis, D. A. (2020). Throw me a lifeline: A professional development program for teacher educators managing the demands from the rapid transition to online teaching. In Ferdig, R.E., Baumgartner, E., Hartshorne, R., Kaplan-Rakowski, R. & Mouza, C. (Eds.) Teaching, Technology, and Teacher Education During the COVID-19 Pandemic: Stories From the Field (pp.517–520). Association for the Advancement of Computing in Education (AACE).
Foulger, T. S., Wetzel, K., & Buss, R. R. (2019). Moving toward a technology infusion approach: considerations for teacher preparation programs. Journal of Digital Learning in Teacher Education. https://doi.org/10.1080/21532974.2019.1568325
Fulmer, S. M., & Frijters, J. C. (2009). A review of self-report and alternative approaches in the measurement of student motivation. Educational Psychology Review, 21(3), 219–246. https://doi.org/10.1007/s10648-009-9107-x
Graham, C. R., Borup, J., Pulham, E., & Larsen, R. (2019). K–12 blended teaching readiness: Model and instrument development. Journal of Research on Technology in Education, 51(3), 239–258. https://doi.org/10.1080/15391523.2019.1586601
Hechter, R. P. (2012). Pre-service teachers’ maturing perceptions of a TPACK framed signature pedagogy in science education. Computers in the Schools, 29(1–2), 53–69.
iNACOL (2009). National Standards for Quality Online Programs. https://aurora-institute.org/wp-content/uploads/national-standards-for-quality-online-programs.pdf
ISTE (2008). ISTE standards for teachers. https://www.iste.org/standards/standards-for-teachers
ISTE (2017). ISTE standards for educators. https://www.iste.org/standards/for-educators
ISTE (2021). Edtech policy maps. https://www.iste.org/edtech-policy-maps
Januszewski, A., Molenda, M., & Harris, P. (Eds.). (2008). Educational technology: a definition with commentary (2nd ed.). Lawrence Erlbaum Associates.
Keating, T., & Evans, E. (2001). Three computers in the back of the classroom: pre-service teachers’ conceptions of technology integration. In R. Carlsen, N. Davis, J. Price, R. Weber, & D. Willis (Eds.), Society for Information Technology and Teacher Education Annual, 2001 (pp. 1671–1676). Association for the Advancement of Computing in Education.
Knafl, K., Deatrick, J., Gallo, A., Holcombe, G., Bakitas, M., Dixon, J., & Grey, M. (2007). The analysis and interpretation of cognitive interviews for instrument development. Research in Nursing & Health, 30(2), 224–234. https://doi.org/10.1002/nur.20195
Knezek, G., Christensen, R., & Furuta, T. (2019). Validation of a teacher educator technology competencies survey. Journal of Technology and Teacher Education, 27(4), 465–498. Retrieved from https://www.learntechlib.org/primary/p/207633/
Koh, J. K. L., & Chai, C. S. (2016). Seven design frames that teachers use when considering technological pedagogical content knowledge (TPACK). Computers & Education, 102, 244–257.
Koehler, M. J., & Mishra, P. (2009). What is technological pedagogical content knowledge? Contemporary Issues in Technology and Teacher Education, 9(1), 60–70.
Linacre, J. M. (2002). What do infit and outfit, mean-square and standardized mean? Rasch Measurement Transactions, 16, 878.
Linacre, J. M. (2015). A user’s guide to Winsteps Ministep Rasch-model computer programs. http://www.winsteps.com/manuals.htm
Martin, B. (2018). Faculty technology beliefs and practices in teacher preparation through a TPACK lens. Education & Information Technologies, 23. https://doi.org/10.1007/s10639-017-9680-4
Messick, S. (1989). Validity. In R. Linn (Ed.), Educational measurement (3rd ed.). American Council on Education.
Mishra, P. (1998). Flexible learning in the periodic system with multiple representations: The design of a hypertext for learning complex concepts in chemistry. (Doctoral dissertation, University of Illinois at Urbana—Champaign). Dissertation Abstracts International, 59(11), 4057. (AAT 9912322).
Moore, G., & Benbasat, I. (1991). Development of an instrument to measure the perceptions of adopting an information technology innovation. Information Systems Research, 2(3), 192–222. https://doi.org/10.1287/isre.2.3.192
Morales-Martinez, G. E., & Santos-Alcantara, M. G. (2015). Alternative empirical directions to evaluate schemata organization and meaning. Advances in Social Sciences Research Journal, 2(9), 51–58. https://doi.org/10.14738/assrj.29.2015
Neumann, K. L., Alvarado-Albertorio, F., & Ramírez-Salgado, A. (2021). Aligning with practice: Examining the effects of a practice-based educational technology course on preservice teachers’ potential to teach with technology. TechTrends 65, 1027–1041. https://doi.org/10.1007/s11528-021-00672-y
NSQOL (2019). The national standards for quality online teaching. Virtual Learning Leadership Alliance and Quality Matters. https://www.nsqol.org/the-standards/quality-online-teaching/
Pierson, M. E. (2001). Technology integration practice as a function of pedagogical expertise. Journal of Research on Computing in Education, 33(4), 413–429.
Polly, D., Mims, C., Shepherd, C., & Inan, F. (2010). Evidence of impact: transforming teacher education with preparing tomorrow’s teachers to teach with technology (PT3) grants. Teaching and Teacher Education, 26, 863–870.
Setari, A. P., Lee, J., & Bradley, K. D. (2016). A psychometric approach to the validation of a student evaluation of teaching instrument. Studies in Educational Evaluation, 51, 77–87. https://doi.org/10.1016/j.stueduc.2016.09.006
Shulman, L. S. (1986). Those who understand: knowledge growth in teaching. Educational Researcher, 15, 4–14.
Shulman, L. S. (1987). Knowledge and teaching. Harvard Educational Review, 57(1), 1–22.
Slykhuis, D. A., Foulger, T. S., Graziano, K. J., Hofer, M., Lee, J. K., & Schmidt-Crawford, D. A. (2020). TETC Professional Development: An Online Program for Teacher Educators. In D. Schmidt-Crawford (Ed.), Proceedings of Society for Information Technology & Teacher Education International Conference (pp.485–489). Online: Association for the Advancement of Computing in Education (AACE). https://www.learntechlib.org/primary/p/215784/
Toland, M. D. (2013). Practical guide for conducting an item-response theory analysis. Journal of Early Adolescence, 34, 120–151.
Tondeur, J., van Braak, J., Sang, G., Voogt, J., Fisher, P., & Ottenbreit-Leftwich, A. (2012). Preparing pre-service teachers to integrate technology in education: a synthesis of qualitative evidence. Computers & Education, 59, 134–144.
Tondeur, J., Pareja Roblin, N., van Braak, J., Voogt, J., & Prestridge, S. (2016). Preparing beginning teachers for technology integration in education: ready for take-off? Technology Pedagogy and Education, 26(2), 157–177.
Trust, T. (2018). 2017 ISTE Standards for Educators: From teaching with technology to using technology to empower learners. Journal of Digital Learning in Teacher Education, 34(1), 1–3. https://doi.org/10.1080/21532974.2017.1398980
U.S. Department of Education, Office of Educational Technology (2016). Future Ready Learning, Reimagining the Role of Technology in Education: 2016 National Education Technology Plan, Washington, D.C. https://tech.ed.gov/files/2015/12/NETP16.pdf
U.S. Department of Education, Office of Educational Technology (2017). Reimagining the Role of Technology in Education: 2017 National Education Technology Plan Update, Washington, D.C. https://tech.ed.gov/files/2017/01/NETP17.pdf
Williams, M. E. (2017). An examination of technology training experiences from teacher candidacy to in-service professional development. Journal of Instructional Pedagogies, 19.
Winslow, J., Fegely, A., Lee, C., & Setari, A. (2021). Continued Rasch development and validation of a standards-based instrument to measure teacher educational technology competencies. In T. Bastiaens (Ed.), Proceedings of EdMedia + Innovate Learning (pp.339–345). United States: Association for the Advancement of Computing in Education (AACE). Retrieved from https://www.learntechlib.org/primary/p/219677/
Zhao, Y. (2003). What teachers should know about technology: perspectives and practices. Information Age Publishing.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Research Involving Human and Animal Rights
Human participants only.
Informed Consent
Yes.
Conflict of Interest
The authors declare that they have no conflict of interest.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Fegely, A., Winslow, J., Lee, CY.(. et al. EdTech Align: a Valid and Reliable Instrument for Measuring Teachers’ EdTech Competencies Aligned to Professional Standards. TechTrends (2023). https://doi.org/10.1007/s11528-022-00824-8
Accepted:
Published:
DOI: https://doi.org/10.1007/s11528-022-00824-8