Journal directory listing - Volume 61 (2016) - Journal of Research in Education Sciences【61(2)】June (Special Issue: Teacher Education: Reflection, Effect, and Prospect)
Directory
(Special Issue) Construction of the Teacher Situational Judgment Test
Author: Tzu-Yang Chao(Department of Educational Psychology and Counseling, National Taiwan Normal University), Jia-Li Huang(Office of Teacher Education and Careers Service, National Taiwan Normal University), Yao-Ting Sung(Department of Educational Psychology and Counseling, National Taiwan Normal University), Hui-Ning Kuo(Research Center for Psychological and Educational Testing, National Taiwan Normal University), Ming-Huei Shiu(Research Center for Psychological and Educational Testing, National Taiwan Normal University)
Vol.&No.：Vol. 61, No. 2
Date：June 2016
Pages：85-117
DOI：10.6209/JORIES.2016.61(2).04
Abstract：
According to the ideal of having multiple measures of teacher education, this study developed the teacher situational judgment test (TSJT) for selecting student teachers. The TSJS was based on the hypothesis that the more similar a student’s thinking is to senior and high-performing teachers, the more likely that student is to become an effective teacher. In Study 1, we collected data from 3,133 intern-teachers and teachers to determine the reliability and validity of the TSJT. On the basis of the analysis of the pilot test and formal test, we retained 75 items. Furthermore, the internal consistency reliability of three of the TSJT booklets was exceeded .7. Additionally, the Rasch model analysis showed that 90.67% of the TSJT items fit the Rasch model. In Study 2, we developed 150 new items to expand the TSJT item bank, and recruited 3,039 college students for an item analysis. Subsequently, we further retained 105 items for the TSJT item bank, totaling 180. The reliability of each booklet was between .47 and .74. For the test validity, the analysis of variances indicated that the TSJT scores of student teachers were significantly higher than those of nonstudent teachers. This provided known-group differences validity for the TSJT. Furthermore, in Study 2, we also conducted a qualitative content analysis to provide additional evidence of the construct validity. This present study discusses the development and practical applications of the TSJT.
Keywords：reliability, validity, teacher situational judgment test, item response theory, qualitative analysis
《Full Text》
References：
- 王文中（2004）。Rasch測量理論與其在教育與心理之應用。教育與心理研究，27（4），637-694。【Wang, W.-C. (2004). Rasch measurement theory and application in education and psychology. Journal of Education & Psychology, 27(4), 637-694.】
- 徐靜嫻（2013）。PBL融入師資培育教學實習課程之個案研究。教育科學研究期刊，58（2），91-121。doi:10.3966/2073753X2013065802004【Hsu, C.-H. (2013). Case study on applying problem-based learning to the student teaching curriculum. Journal of Research in Education Sciences, 58(2), 91-121. doi:10.3966/2073753X2013065802004】
- 張芬芬（2010）。質性資料分析的五步驟：在抽象階梯上爬升。初等教育期刊，35，87-120。【Chang, F.-F. (2010). The five steps of qualitative data analysis: Climbing up a ladder of abstraction. Journal of Elementary Education, 35, 87-120.】
- 教育部（2012）。中華民國師資培育白皮書。臺北市：作者。【Ministry of Education. (2012). White paper for teacher education of R.O.C. Taipei, Taiwan: Author.】
- 陳美玉（2005）。師資生個人知識管理及其對專業學習與發展影響之研究。師大學報：教育類，50（2），181-202。doi:10.3966/2073753X2005105002010【Chen, M.-Y. (2005). Teachers college students’ personal knowledge management and it’s impact on professional learning and development. Journal of National Taiwan Normal University: Education, 50(2), 181-202. doi:10. 3966/2073753X2005105002010】
» More
- 王文中（2004）。Rasch測量理論與其在教育與心理之應用。教育與心理研究，27（4），637-694。【Wang, W.-C. (2004). Rasch measurement theory and application in education and psychology. Journal of Education & Psychology, 27(4), 637-694.】
- 徐靜嫻（2013）。PBL融入師資培育教學實習課程之個案研究。教育科學研究期刊，58（2），91-121。doi:10.3966/2073753X2013065802004【Hsu, C.-H. (2013). Case study on applying problem-based learning to the student teaching curriculum. Journal of Research in Education Sciences, 58(2), 91-121. doi:10.3966/2073753X2013065802004】
- 張芬芬（2010）。質性資料分析的五步驟：在抽象階梯上爬升。初等教育期刊，35，87-120。【Chang, F.-F. (2010). The five steps of qualitative data analysis: Climbing up a ladder of abstraction. Journal of Elementary Education, 35, 87-120.】
- 教育部（2012）。中華民國師資培育白皮書。臺北市：作者。【Ministry of Education. (2012). White paper for teacher education of R.O.C. Taipei, Taiwan: Author.】
- 陳美玉（2005）。師資生個人知識管理及其對專業學習與發展影響之研究。師大學報：教育類，50（2），181-202。doi:10.3966/2073753X2005105002010【Chen, M.-Y. (2005). Teachers college students’ personal knowledge management and it’s impact on professional learning and development. Journal of National Taiwan Normal University: Education, 50(2), 181-202. doi:10. 3966/2073753X2005105002010】
- 黃源河、符碧真（2009）。一樣軌跡兩樣情：美國與我國師資培育機構轉型與教師質量之比較。教育科學研究期刊，54（4），171-200。doi:10.3966/2073753X2009125404007【Huang, Y. R., & Fwu, B.-C. (2009). Similar routes but different paths: teacher preparation, teacher quantity and quality in the U.S. and Taiwan. Journal of Research in Education Sciences, 54(4), 171-200. doi:10.3966/ 2073753X2009125404007】
- 黃嘉莉（2013）。標準本位師資培育理念與實踐。臺北市：國立臺灣師範大學出版中心。【Huang, J.-L. (2013). The ideas and practice of standards-based teacher education. Taipei, Taiwan: National Taiwan Normal University Press.】
- 黃嘉莉、武佳瀅（2015）。我國教育實習學生知覺教師專業知能習得、運用與重要之研究。教育科學研究期刊，60（2），1-32。doi:10.6209/JORIES.2015.60(2).01【Huang, J.-L., & Wu, J.-Y. (2015). Student teacher perception of teacher education and professional knowledge acquisition, use, and importance in Taiwan. Journal of Research in Education Sciences, 60(2), 1-32. doi:10. 6209/JORIES.2015.60(2).01】
- 黃嘉莉、趙子揚、宋曜廷（2013）。教師情境判斷測驗編製之理論基礎與實踐。載於吳清基、黃嘉莉（主編），雲端時代的師資培育（pp. 97-124）。臺北市：中華民國師範教育學會。【Huang, J.-L., Chao, T.-Y., & Sung, Y.-T. (2013). The theory and practice for compiling teacher situational judgment test. In C.-J. Wu & J.-L. Huang (Eds.), Teacher education in E-cloud era (pp. 97-124). Taipei, Taiwan: Teacher Education Society of R.O.C.】
- 楊國樞、陸洛（2005）。社會取向自我實現者與個人取向自我實現者的心理特徵：概念分析與實徵衡鑑。本土心理學研究，23，71-143。【Yang, K.-S., & Lu, L. (2005). Social-and individual-oriented self-actualizers: Conceptual analysis and empirical assessment of their psychological characteristics. Indigenous Psychological Research in Chinese Societies, 23, 71-143.】
- 歐宗霖、邱皓政、孫國勛（2013）。當測驗碰上情境：情境判斷測驗與自陳量表在量表發展上之比較。測驗學刊，60（2），239-262。【Ou, T.-L., Chiou, H.-J., & Sun, K.-S. (2013). When test met scenario: The comparison between situational judgment test and self-report scale on scale development. Psychological Testing, 60(2), 239-262.】
- 鄭育文、陳柏熹、宋曜廷、陳信豪、蕭孟莛（2014）。電腦化適性職涯性向測驗編製研究。教育心理學報，46（2），271-288。doi:10.6251/BEP.20140516【Cheng, Y.-W., Chen, P.-H., Sung, Y.-T., Chen, S.-H., & Hsiao, M.-T. (2014). Construction of the computerized adaptive career aptitude test for junior high school students. Bulletin of Education Psychology, 46(2), 271-288. doi:10.6251/BEP.20140516】
- 賴保禎、賴美玲（2003）。賴氏人格測驗。臺北市：千華。【Lai, B.-C., & Lai, M.-L. (2003). Lai’s personality inventory. Taipei, Taiwan: Chien Hua.】
- Arenson, E. A. (1999, March). Statistical linkages between state education assessments and the national assessment of educational progress. Paper presented at the annual meeting of the Sacramento Statistical Association, Sacramento, CA.
- Chan, D., & Schmitt, N. (1997). Video-based versus paper-and-pencil method of assessment in situational judgment tests: Subgroup differences in test performance and face validity perceptions. Journal of Applied Psychology, 82(1), 143-159. doi:10.1037/0021-9010.82.1.143
- Chan, D., & Schmitt, N. (2002). Situational judgment and job performance. Human Performance, 15(3), 233-254. doi:10.1207/S15327043HUP1503_01
- Cheng, C.-S., & Bailey, R. A. (1991). Optimality of some two-associate-class partially balanced incomplete-block designs. The Annals of Statistics, 19(3), 1667-1671. doi:10.1214/aos/1176348270
- Clevenger, J., Pereira, G. M., Wiechmann, D., Schmitt, N., & Harvey, V. S. (2001). Incremental validity of situational judgment tests. Journal of Applied Psychology, 86(3), 410-417. doi:10. 1037/0021-9010.86.3.410
- Cohen, J. (1977). Statistical power analysis for the behavioral sciences. New York, NY: Academic Press.
- Cohen, D. K., Raudenbush, S. W., & Ball, D. L. (2003). Resources, instruction, and research. Educational Evaluation and Policy Analysis, 25(2), 119-142. doi:10.3102/01623737025002119
- Conner, L. A. (2007). Evaluation of the situational judgment test (Unpublished master’s thesis). University of North Texas, TX.
- de Meijer, L. A. L., Born, M. Ph., van Zielst, J., & van der Molen, H. T. (2010). Construct-driven development of a video-based situational judgment test for integrity: A study in a muilti-Ethnic polic setting. European Psychologist, 15(3), 229-236. doi:10.1027/1016-9040/a000027
- Educational Testing Service. (2009). 2009—10 The Praxis SeriesTM Bulletin. Retrieved from http://www.ets.org/praxis
- Fenstermacher, G. D., & Richardson, V. (2005). On making determinations of quality in teaching. Teachers College Record, 107(1), 186-213. doi:10.1111/j.1467-9620.2005.00462.x
- Hoban, G. F. (2000). Using a reflective framework to study teaching-learning relationships. Reflective Practice, 1(2), 165-182. doi:10.1080/713693148
- Kennedy, M. M. (2006). From teacher quality to quality teaching. Educational Leadership, 63(6), 14-19.
- Koster, B., Dengerink, J., Korthagen, F., & Lunenberg, M. (2008). Teacher educators working on their own professional development: Goals, activities and outcomes of a project for the professional development of teacher educators. Teachers and Teaching: Theory and Practice, 14(5-6), 567-587. doi:10.1080/13540600802571411
- Labaree, D. F. (2000). On the nature of teaching and teacher education: Difficult practices that look easy. Journal of Teacher Education, 51(3), 228-233. doi:10.1177/0022487100051003011
- Layne, H. B., & Boyles, R. D. (1997). Characteristics of ‘good’ teachers. Research and Reflections, 3(2), 17-39.
- Lievens, F., Peeters, H., & Schollaert, E. (2008). Situational judgment tests: A review of recent research. Personnel Review, 37(4), 426-441. doi:10.1108/00483480810877598
- Lievens, F., & Sackett, P. R. (2006). Video-based versus written situational judgment tests: A comparison in terms of predictive validity. Journal of Applied Psychology, 91(5), 1181-1188. doi:10.1037/0021-9010.91.5.1181
- Lievens, F., Sackett, P. R., & Buyse, T. (2009). The effect of response instructions on situational judgment test performance and validity in a high-stakes context. Journal of Applied Psychology, 94(4), 1095-1101. doi:10.1037/a0014628.
- Masters, G. (1982). A Rasch model for partial credit scoring. Psychometrika, 47(2), 149-174.
- McDaniel, M. A., Morgeson, F. P., Finnegan, E. B., Campion, M. A., & Braverman, E. P. (2001). Use of situational judgement tests to predict job performance: A clarification of the literature. Journal of Applied Psychology, 86(4), 730-740. doi:10.1037/0021-9010.86.4.730.
- McDaniel, M. A., & Nguyen, N. T. (2001). Situational judgment tests: A review of practice and constructs assessed. International Journal of Selection and Assessment, 9(1&2), 103-113. doi: 10.1111/1468-2389.00167
- McDaniel, M. A., & Whetzel, D. L. (2005). Situational judgment test research: Informing the debate on practical intelligence theory. Intelligence, 33(5), 515-525. doi:10.1016/j.intell.2005.02.001
- Miller, W. L., & Crabtree, B. F. (1999). Doing qualitative research (2nd ed.). London, UK: Sage.
- Motowidlo, S. J., Dunnette, M. D., & Carter, G. W. (1990). An alternative selection procedure: The low-fidelity simulation. Journal of Applied Psychology, 75(6), 640-647. doi:10.1037/0021-9010. 75.6.640
- Nguyen, N. T., & McDaniel, M. A. (2003). Response instructions and racial differences in a situational judgment test. Applied H.R.M. Research, 8(1), 33-44.
- O’Connell, M. S., Hartman, N. S., McDaniel, M. A., Grubb, W. L., & Lawrence, A. (2007). Incremental validity of situational judgment tests for task and contextual job performance. International Journal of Selection and Assessment, 15(1), 19-29. doi:10.1111/j.1468-2389.2007. 00364.x
- Ogden, D. H. (1994). Characteristics of good/effective teachers: Gender differences in student descriptors. Retrieved from ERIC database. (ED383657)
- Olson-Buchanan, J. B., Drasgow, F., Moberg, P. J., Mead, A. D., Keenan, P. A., & Donovan, M. A. (1998). Interactive video assessment of conflict resolution skills. Personnel Psychology, 51(1), 1-24. doi:10.1111/j.1744-6570.1998.tb00714.x
- Ployhart, R. E., Porr, W., & Ryan, A. M. (2004, April). New development in SJTs: Scoring, coaching and incremental validity. Paper presented at the 19th annual convention of the Society for Industrial and Organizational Psychology, Chicago, IL.
- Pring, P. (2004). Teaching and the good teacher. In F. Inglis (Ed.), Education and the good society (pp. 75-87). London, UK: Palgrave Macmillan.
- Shanoski, L. A., & Hranitz, J. R. (1991). A foundation for excellence in teaching. Retrieved from ERIC database. (ED356212)
- Shulman, L. S. (1987). Sounding an alarm: A reply to Sockett. Harvard Educational Review, 57(4), 473-484. doi:10.17763/haer.57.4.47w18u3586513887
- Speicher, A. D., & Schurter, W. A. (1981). Instructional improvement and student learning through good teacher evaluation. Retrieved from ERIC database. (ED239380)
- Verhelst, N. D., & Verstralen, H. H. F. M. (2008). Some considerations on the partial credit model. Psicológica: Revista de metodología y psicología experimental, 29(2), 229-254.
- Wagner, R. K., & Sternberg, R. J. (1985). Practical intelligence in real-world pursuits: The role of tacit knowledge. Journal of Personality and Social Psychology, 49(2), 436-458. doi:10.1037/ 0022-3514.49.2.436
- Weekley, J. A., & Jones, C. (1997). Video-based situational testing. Personnel Psychology, 50(1), 25-49. doi:10.1111/j.1744-6570.1997.tb00899.x
- Weekley, J. A., & Jones, C. (1999). Further studies of situational tests. Personnel Psychology, 52(3), 679-700. doi:10.1111/j.1744-6570.1999.tb00176.x
- Wenzlaff, T. L. (1998). Dispositions and portfolio development: Is there a connection? Education, 118(4), 564-572.
- Whetzel, D. L., & McDaniel, M. A. (2009). Situational judgment tests: An overview of current research. Human Resource Management Review, 19(3), 188-202. doi:10.1016/j.hrmr.2009.03.007
- Wright, B. D., & Linacre, J. M. (1994). Reasonable mean-square fit values. Rasch Measurement: Transactions of the Rasch Measurement SIG, 8(3), 370.
- Wu, M. L., Adams, R. J., & Wilson, M. R. (1998). ConQuest [Computer software and manual]. Camberwell, AU: Australian Council for Educational Research.
| APA Format | Chao, T.-Y., Huang, J.-L., Sung, Y.-T., Kuo, H.-N., & Shiu, M.-H. (2016). Construction of the Teacher Situational Judgment Test. Journal of Research in Education Sciences, 61(2), 85-117. doi:10.6209/JORIES.2016.61(2).04 |
|---|