Skip to main content

Advertisement

SpringerLink
Log in

EdTech Align: a Valid and Reliable Instrument for Measuring Teachers’ EdTech Competencies Aligned to Professional Standards

  • Original Paper
  • Published:
TechTrends Aims and scope Submit manuscript

Abstract

The purpose of this article was to share a standards-aligned instrument designed for evaluating teachers’ educational technology competencies. A survey derived from multiple sets of popular educational technology standards was developed and administered in four phases to a total of 184 teachers. Constructive response, cognitive interviews, and Rasch empirical analyses were used to refine the instrument. These processes resulted in an instrument with evidence to indicate validity and reliability that aligned to the International Society for Technology in Education (ISTE) Standards for Educators, Association for Educational Communications and Technology (AECT) Standards, Association for the Advancement of Computing Education (AACE) Teacher Educator Technology Competencies, and Virtual Learning Leadership (VLL), Quality Matters (QM), and the Digital Learning Collaborative (DLC) National Standards for Quality Online Teaching. This study provides an instrument that may be used by educator preparation providers, school districts, accreditation bodies, or other educational institutions to measure educators’ edtech competencies derived from professional standards.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  • AECT (Association for Educational Communications and Technology) (1989). Guidelines for the accreditation of programs in educational communications and information technologies. Washington, DC.

  • AECT (Association for Educational Communications and Technology) (2012). AECT Standards. Retrieved from https://www.aect.org/docs/AECTstandards2012.pdf

  • Ayad, F. I., & Ajrami, S. J. (2017). The degree of implementing ISTE standards in technical education colleges of Palestine. The Turkish Journal of Educational Technology, 16(2), 107–118.

    Google Scholar 

  • Bakir, N. (2016). Technology and teacher education: A brief glimpse of the research and practice that have shaped the field. TechTrends, 60, 21–29. https://doi.org/10.1007/s11528-015-0013-4

  • Bond, T. G., & Fox, C. M. (2007). Applying the Rasch Model: fundamental measurement in the human sciences (2nd ed.). Lawrence Erlbaum Associates.

    Google Scholar 

  • Bradley, K. D., & Bradley, J. W. (2006). Challenging the validity of higher education course evaluations. Journal of College Teaching & Learning, 3(8), 63–76.

    Google Scholar 

  • Bush, B., & Hurlbut, A. (2020). Measuring digital literacy: Instrument development to assess technology usage and integration for continuous program improvement. In D. Schmidt-Crawford (Ed.), Proceedings of Society for Information Technology & Teacher Education International Conference (pp. 981–985). Online: Association for the Advancement of Computing in Education (AACE). https://www.learntechlib.org/primary/p/215849/

  • Council for the Accreditation of Educator Preparation (2022). 2022 Initial Level Standards. https://caepnet.org/~/media/Files/caep/standards/2022-initial-standards-1-pager-final.pdf?la=en

  • Chai, C. S., Ling Koh, H., & Teo, Y. H. (2019). Enhancing and modeling teachers’ design beliefs and efficacy of technological pedagogical content knowledge for 21st century quality learning. Journal of Educational Computing Research, 57(2), 360–384.

    Article  Google Scholar 

  • Cherner, T., Fegely, A. & Gleasman, C. L. (2021). We need to talk about the ISTE Students standards: A critical analysis. In D.C. Gibson & M.N. Ochoa (Eds.), Research Highlights in Technology and Teacher Education (pp. 85–92). AACE. https://www.learntechlib.org/p/220334/

  • Douglas, J. A. (2011). Competency for quality online teaching. [Doctoral dissertation, University of Virginia]. ProQuest Dissertations & Theses. https://www.proquest.com/docview/875240837

  • Faison, C. (1994). Integrating technology into teacher education: a modular approach. In J. Willis, B. Robin, & D. Willis (Eds.), Technology and Teacher Education Annual, 1994 (pp. 169–171). Association for the Advancement of Computing in Education.

    Google Scholar 

  • Foulger, T. S., Buss, R. R., Wetzel, K., & Lindsey, L. (2015). Instructors’ growth in TPACK: teaching technology-infused methods courses to preservice teachers. Journal of Digital Learning in Teacher Education, 31(4), 134–147.

    Article  Google Scholar 

  • Foulger, T. S., Graziano, K. J., Schmidt-Crawford, D., & Slykhuis, D. A. (2017). Teacher educator technology competencies. Journal of Technology and Teacher Education, 25(4), 413–448. Society for Information Technology & Teacher Education. Retrieved from https://www.learntechlib.org/p/181966/

  • Foulger, T. S., Graziano, K. J., Schmidt-Crawford, D. A., & Slykhuis, D. A. (2020). Throw me a lifeline: A professional development program for teacher educators managing the demands from the rapid transition to online teaching. In Ferdig, R.E., Baumgartner, E., Hartshorne, R., Kaplan-Rakowski, R. & Mouza, C. (Eds.) Teaching, Technology, and Teacher Education During the COVID-19 Pandemic: Stories From the Field (pp.517–520). Association for the Advancement of Computing in Education (AACE).

  • Foulger, T. S., Wetzel, K., & Buss, R. R. (2019). Moving toward a technology infusion approach: considerations for teacher preparation programs. Journal of Digital Learning in Teacher Education. https://doi.org/10.1080/21532974.2019.1568325

    Article  Google Scholar 

  • Fulmer, S. M., & Frijters, J. C. (2009). A review of self-report and alternative approaches in the measurement of student motivation. Educational Psychology Review, 21(3), 219–246. https://doi.org/10.1007/s10648-009-9107-x

    Article  Google Scholar 

  • Graham, C. R., Borup, J., Pulham, E., & Larsen, R. (2019). K–12 blended teaching readiness: Model and instrument development. Journal of Research on Technology in Education, 51(3), 239–258. https://doi.org/10.1080/15391523.2019.1586601

  • Hechter, R. P. (2012). Pre-service teachers’ maturing perceptions of a TPACK framed signature pedagogy in science education. Computers in the Schools, 29(1–2), 53–69.

  • iNACOL (2009). National Standards for Quality Online Programs. https://aurora-institute.org/wp-content/uploads/national-standards-for-quality-online-programs.pdf

  • ISTE (2008). ISTE standards for teachers. https://www.iste.org/standards/standards-for-teachers

  • ISTE (2017). ISTE standards for educators. https://www.iste.org/standards/for-educators

  • ISTE (2021). Edtech policy maps. https://www.iste.org/edtech-policy-maps

  • Januszewski, A., Molenda, M., & Harris, P. (Eds.). (2008). Educational technology: a definition with commentary (2nd ed.). Lawrence Erlbaum Associates.

  • Keating, T., & Evans, E. (2001). Three computers in the back of the classroom: pre-service teachers’ conceptions of technology integration. In R. Carlsen, N. Davis, J. Price, R. Weber, & D. Willis (Eds.), Society for Information Technology and Teacher Education Annual, 2001 (pp. 1671–1676). Association for the Advancement of Computing in Education.

    Google Scholar 

  • Knafl, K., Deatrick, J., Gallo, A., Holcombe, G., Bakitas, M., Dixon, J., & Grey, M. (2007). The analysis and interpretation of cognitive interviews for instrument development. Research in Nursing & Health, 30(2), 224–234. https://doi.org/10.1002/nur.20195

    Article  Google Scholar 

  • Knezek, G., Christensen, R., & Furuta, T. (2019). Validation of a teacher educator technology competencies survey. Journal of Technology and Teacher Education, 27(4), 465–498. Retrieved from https://www.learntechlib.org/primary/p/207633/

  • Koh, J. K. L., & Chai, C. S. (2016). Seven design frames that teachers use when considering technological pedagogical content knowledge (TPACK). Computers & Education, 102, 244–257.

    Article  Google Scholar 

  • Koehler, M. J., & Mishra, P. (2009). What is technological pedagogical content knowledge? Contemporary Issues in Technology and Teacher Education, 9(1), 60–70.

    Google Scholar 

  • Linacre, J. M. (2002). What do infit and outfit, mean-square and standardized mean? Rasch Measurement Transactions, 16, 878.

    Google Scholar 

  • Linacre, J. M. (2015). A user’s guide to Winsteps Ministep Rasch-model computer programs. http://www.winsteps.com/manuals.htm

  • Martin, B. (2018). Faculty technology beliefs and practices in teacher preparation through a TPACK lens. Education & Information Technologies, 23. https://doi.org/10.1007/s10639-017-9680-4

  • Messick, S. (1989). Validity. In R. Linn (Ed.), Educational measurement (3rd ed.). American Council on Education.

    Google Scholar 

  • Mishra, P. (1998). Flexible learning in the periodic system with multiple representations: The design of a hypertext for learning complex concepts in chemistry. (Doctoral dissertation, University of Illinois at Urbana—Champaign). Dissertation Abstracts International, 59(11), 4057. (AAT 9912322).

  • Moore, G., & Benbasat, I. (1991). Development of an instrument to measure the perceptions of adopting an information technology innovation. Information Systems Research, 2(3), 192–222. https://doi.org/10.1287/isre.2.3.192

    Article  Google Scholar 

  • Morales-Martinez, G. E., & Santos-Alcantara, M. G. (2015). Alternative empirical directions to evaluate schemata organization and meaning. Advances in Social Sciences Research Journal, 2(9), 51–58. https://doi.org/10.14738/assrj.29.2015

    Article  Google Scholar 

  • Neumann, K. L., Alvarado-Albertorio, F., & Ramírez-Salgado, A. (2021). Aligning with practice: Examining the effects of a practice-based educational technology course on preservice teachers’ potential to teach with technology. TechTrends 65, 1027–1041. https://doi.org/10.1007/s11528-021-00672-y

  • NSQOL (2019). The national standards for quality online teaching. Virtual Learning Leadership Alliance and Quality Matters. https://www.nsqol.org/the-standards/quality-online-teaching/

  • Pierson, M. E. (2001). Technology integration practice as a function of pedagogical expertise. Journal of Research on Computing in Education, 33(4), 413–429.

    Article  Google Scholar 

  • Polly, D., Mims, C., Shepherd, C., & Inan, F. (2010). Evidence of impact: transforming teacher education with preparing tomorrow’s teachers to teach with technology (PT3) grants. Teaching and Teacher Education, 26, 863–870.

    Article  Google Scholar 

  • Setari, A. P., Lee, J., & Bradley, K. D. (2016). A psychometric approach to the validation of a student evaluation of teaching instrument. Studies in Educational Evaluation, 51, 77–87. https://doi.org/10.1016/j.stueduc.2016.09.006

    Article  Google Scholar 

  • Shulman, L. S. (1986). Those who understand: knowledge growth in teaching. Educational Researcher, 15, 4–14.

    Article  Google Scholar 

  • Shulman, L. S. (1987). Knowledge and teaching. Harvard Educational Review, 57(1), 1–22.

    Article  Google Scholar 

  • Slykhuis, D. A., Foulger, T. S., Graziano, K. J., Hofer, M., Lee, J. K., & Schmidt-Crawford, D. A. (2020). TETC Professional Development: An Online Program for Teacher Educators. In D. Schmidt-Crawford (Ed.), Proceedings of Society for Information Technology & Teacher Education International Conference (pp.485–489). Online: Association for the Advancement of Computing in Education (AACE). https://www.learntechlib.org/primary/p/215784/

  • Toland, M. D. (2013). Practical guide for conducting an item-response theory analysis. Journal of Early Adolescence, 34, 120–151.

    Article  Google Scholar 

  • Tondeur, J., van Braak, J., Sang, G., Voogt, J., Fisher, P., & Ottenbreit-Leftwich, A. (2012). Preparing pre-service teachers to integrate technology in education: a synthesis of qualitative evidence. Computers & Education, 59, 134–144.

    Article  Google Scholar 

  • Tondeur, J., Pareja Roblin, N., van Braak, J., Voogt, J., & Prestridge, S. (2016). Preparing beginning teachers for technology integration in education: ready for take-off? Technology Pedagogy and Education, 26(2), 157–177.

    Article  Google Scholar 

  • Trust, T. (2018). 2017 ISTE Standards for Educators: From teaching with technology to using technology to empower learners. Journal of Digital Learning in Teacher Education, 34(1), 1–3. https://doi.org/10.1080/21532974.2017.1398980

  • U.S. Department of Education, Office of Educational Technology (2016). Future Ready Learning, Reimagining the Role of Technology in Education: 2016 National Education Technology Plan, Washington, D.C. https://tech.ed.gov/files/2015/12/NETP16.pdf

  • U.S. Department of Education, Office of Educational Technology (2017). Reimagining the Role of Technology in Education: 2017 National Education Technology Plan Update, Washington, D.C. https://tech.ed.gov/files/2017/01/NETP17.pdf

  • Williams, M. E. (2017). An examination of technology training experiences from teacher candidacy to in-service professional development. Journal of Instructional Pedagogies, 19.

  • Winslow, J., Fegely, A., Lee, C., & Setari, A. (2021). Continued Rasch development and validation of a standards-based instrument to measure teacher educational technology competencies. In T. Bastiaens (Ed.), Proceedings of EdMedia + Innovate Learning (pp.339–345). United States: Association for the Advancement of Computing in Education (AACE). Retrieved from https://www.learntechlib.org/primary/p/219677/

  • Zhao, Y. (2003). What teachers should know about technology: perspectives and practices. Information Age Publishing.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Educational Studies Spadoni College of Education & Social Sciences, Coastal Carolina University, P.O. Box 261954, Conway, SC, 29528, USA

    Alex Fegely, Joseph Winslow & Cheng-Yuan (Corey) Lee

  2. Department of Education Sciences and Organizations Spadoni College of Education & Social Sciences, Coastal Carolina University, P.O. Box 261954, Conway, SC, 29528, USA

    Anthony P. Setari

Authors
  1. Alex Fegely
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Joseph Winslow
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Cheng-Yuan (Corey) Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Anthony P. Setari
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alex Fegely.

Ethics declarations

Research Involving Human and Animal Rights

Human participants only.

Informed Consent

Yes. 

Conflict of Interest

The authors declare that they have no conflict of interest. 

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fegely, A., Winslow, J., Lee, CY.(. et al. EdTech Align: a Valid and Reliable Instrument for Measuring Teachers’ EdTech Competencies Aligned to Professional Standards. TechTrends (2023). https://doi.org/10.1007/s11528-022-00824-8

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11528-022-00824-8

Keywords

Search

Navigation