Journal directory listing - Volume 63 (2018) - Journal of Research in Education Sciences【63(4)】December (Special Issue: The Concepts and Practices for Achieving Literacy)
Directory
(Special Issue) Development and Validation of an Online Assessment Module for Mandarin Reading Literacy for Middle School Students of the 12-Year Basic Education Program
Author: Pei-Jung Hsieh (Research Center for Testing and Assessment, National Academy for Educational Research)
Vol.&No.：Vol. 63, No.4
Date：December 2018
Pages：193-228
DOI：10.6209/JORIES.201812_63(4).0007
Abstract：
Reading is one of the key elements of the 12-year basic education curriculum guidelines for elementary schools, junior high schools, and general senior high schools: Language and Literature Category － Chinese Language and Literature. This study developed an online reading assessment module in line with the new curriculum guidelines. A literacy-based assessment module was established to construct an assessment framework, text selection or writing principles, item writing principles, and reading literacy-oriented items. The psychometric characteristics of the items were also investigated to support the validity of the module. The principles and items were validated through the triangulation method. By conducting assessment framework meetings and item writing workshops, a 3-dimensional assessment framework, 17 indicators, 5 text writing principles, and 10 item writing principles were obtained. The Online Assessment for Science Literacy platform was adopted for 23 testlets with 201 items for pilot test administration. The participants were 1,685 junior high school students from 58 schools in Taiwan. Item difficulty, discrimination, and infit mean square statistics were analyzed using the CTT and TAM packages in R. The average discrimination of 17 testlets (73.91%) was more than .30. The average p-value of multiple choice items ranged from .45 to .74. The average discrimination of half of the constructed response items was .37 and the average p-value was .24, which revealed high discrimination and high difficulty. The sample items were also provided as a reference for preparing assessment designs for schools in the future.
Keywords：literacy-based assessment, online assessment, reading comprehension, secondary education
《Full Text》
References：
- 大學入學考試中心（2018）。107學年度學科能力測驗試題國文考科（選擇題）。取自http://www. ceec.edu.tw/AbilityExam/AbilityExamPaper/107SAT_Paper/01-107%E5%AD%B8%E6%B8%AC%E5%9C%8B%E6%96%87%E5%AE%9A%E7%A8%BF.pdf【College Entrance Examination Center. (2018). 2018 general scholastic ability test: Mandarin Chinese. Retrieved from http://www.ceec.edu.tw/AbilityExam/AbilityExamPaper/107SAT_Paper/01-107%E5%AD%B8%E6%B8 %AC%E5%9C%8B%E6%96%87%E5%AE%9A%E7%A8%BF.pdf】
- 中央社（2016，9月13日）。研究曝光 糖業者掩飾糖不利心臟健康。取自http://www.cna.com. tw/news/ahel/201609130487-1.aspx【Central News Agency. (2016, September 13). Sugar industry and coronary heart disease research. Retrieved from http://www.cna.com.tw/news/ahel/201609130487-1.aspx】
- 任宗浩（2018）。素養導向評量的界定與實踐。載於蔡清華（主編），課程協作與實踐第二輯（pp. 75-82）。臺北市：教育部協作中心。【Jen, T.-H. (2018). Literacy-based assessment: Principles and practice. In C.-H. Tsai (Ed.), Curriculum coordination in practice (Vol. II, pp. 75-82). Taipei, Taiwan: Coordination Center for Teacher Professional Development, Curriculum, Instruction and Assessment.】
- 佘曉清、林煥祥（主編）（2017）。PISA 2015臺灣學生的表現。新北市：心理。【She, H.-C., & Lin, H.-S. (Eds.). (2017). Taiwan student performance on PISA 2015. New Taipei City, Taiwan: Psychological.】
- 侯佩君、杜素豪、廖培珊、洪永泰、章英華（2008）。臺灣鄉鎮市區類型之研究：「臺灣社會變遷基本調查第五期計畫之抽樣分層效果分析」。調查研究─方法與應用，23，7-32。doi:10.7014/TCYCFFYYY.200804.0007【Hou, P.-C., Tu, S.-H., Liao, P.-S., Hung, Y.-T., & Chang, Y.-H. (2008). The typology of townships in Taiwan: The analysis of sampling stratification of the 2005-2006 “Taiwan Social Change Survey”. Survey Research − Method and Application, 23, 7-32. doi:10.7014/TCYCFFYYY.200804.0007】
» More
- 大學入學考試中心（2018）。107學年度學科能力測驗試題國文考科（選擇題）。取自http://www. ceec.edu.tw/AbilityExam/AbilityExamPaper/107SAT_Paper/01-107%E5%AD%B8%E6%B8%AC%E5%9C%8B%E6%96%87%E5%AE%9A%E7%A8%BF.pdf【College Entrance Examination Center. (2018). 2018 general scholastic ability test: Mandarin Chinese. Retrieved from http://www.ceec.edu.tw/AbilityExam/AbilityExamPaper/107SAT_Paper/01-107%E5%AD%B8%E6%B8 %AC%E5%9C%8B%E6%96%87%E5%AE%9A%E7%A8%BF.pdf】
- 中央社（2016，9月13日）。研究曝光 糖業者掩飾糖不利心臟健康。取自http://www.cna.com. tw/news/ahel/201609130487-1.aspx【Central News Agency. (2016, September 13). Sugar industry and coronary heart disease research. Retrieved from http://www.cna.com.tw/news/ahel/201609130487-1.aspx】
- 任宗浩（2018）。素養導向評量的界定與實踐。載於蔡清華（主編），課程協作與實踐第二輯（pp. 75-82）。臺北市：教育部協作中心。【Jen, T.-H. (2018). Literacy-based assessment: Principles and practice. In C.-H. Tsai (Ed.), Curriculum coordination in practice (Vol. II, pp. 75-82). Taipei, Taiwan: Coordination Center for Teacher Professional Development, Curriculum, Instruction and Assessment.】
- 佘曉清、林煥祥（主編）（2017）。PISA 2015臺灣學生的表現。新北市：心理。【She, H.-C., & Lin, H.-S. (Eds.). (2017). Taiwan student performance on PISA 2015. New Taipei City, Taiwan: Psychological.】
- 侯佩君、杜素豪、廖培珊、洪永泰、章英華（2008）。臺灣鄉鎮市區類型之研究：「臺灣社會變遷基本調查第五期計畫之抽樣分層效果分析」。調查研究─方法與應用，23，7-32。doi:10.7014/TCYCFFYYY.200804.0007【Hou, P.-C., Tu, S.-H., Liao, P.-S., Hung, Y.-T., & Chang, Y.-H. (2008). The typology of townships in Taiwan: The analysis of sampling stratification of the 2005-2006 “Taiwan Social Change Survey”. Survey Research − Method and Application, 23, 7-32. doi:10.7014/TCYCFFYYY.200804.0007】
- 唐淑華、蔡孟寧、林烘煜（2015）。多文本課外閱讀對增進國中學生理解歷史主題之研究：以「外侮」主題為例。教育科學研究期刊，60（3），63-94。doi:10.6209/JORIES.2015.60(3).03【Tang, S.-H., Tsai, M.-N., & Lin, H.-Y. (2015). Using multiple texts to foster junior high students’ historical understanding. Journal of Research in Education Sciences, 60(3), 63-94. doi:10.6209/JORIES.2015.60(3).03】
- 國立臺灣師範大學心理與教育測驗研究發展中心（2014）。國中教育會考成立背景。取自https://cap.nace.edu.tw/background.html【Research Center for Psychological and Educational Testing, National Taiwan Normal University. (2014). Overview of the comprehensive assessment program for junior high school students. Retrieved from https://cap.nace.edu.tw/background.html】
- 國家教育研究院（2017a）。十二年國教各領域／科目課程綱要課程手冊初稿（第四波更新）：語文領域─國語文課程手冊。新北市：作者。【National Academy for Educational Research. (2017a). Instructor’s manual of 12-year basic education curriculum guidelines: Mandarin Chinese (4th ed.). New Taipei City, Taiwan: Author.】
- 國家教育研究院（2017b）。學力檢測公布系統。取自http://rap.naer.edu.tw/【National Academy for Educational Research. (2017b). Countywide student assessment program. Retrieved from http://rap.naer.edu.tw/】
- 教育部（2014）。十二年國民基本教育課程綱要總綱。臺北市：作者。【Ministry of Education. (2014). 12-year basic education curriculum guidelines. Taipei, Taiwan: Author.】
- 教育部（2018a）。教育部令。行政院公報，24（18）。取自https://gazette.nat.gov.tw/egFront/ detail.do?metaid=96244&log=detailLog【Ministry of Education. (2018a). Ministry of education order. The Executive Yuan Gazette Online, 24(18). Retrieved from https://gazette.nat.gov.tw/egFront/detail.do?metaid=96244&log=detailLog】
- 教育部（2018b）。十二年國民基本教育課程綱要國民中小學暨普通型高級中等學校：語文領域─國語文。臺北市：作者。【Ministry of Education. (2018b). 12-year basic education curriculum guidelines for elementary schools, junior high schools, and general senior high schools: Language and literature category − Chinese language and literature. Taipei, Taiwan: Author.】
- 陳美芳（2014）。精進大學寫作指引。臺北市：國立臺灣師範大學。【Chen, M.-F. (2014). Intensive writing guidelines for university students. Taipei, Taiwan: National Taiwan Normal University.】
- 陳雅慧、王韻齡（2018）。2018《親子天下》縣市教育力調查：新課綱上路前夕，行政減量、課綱準備還需努力。取自https://www.parenting.com.tw/article/5077684-2018%E3%80%8A %E8%A6%AA%E5%AD%90%E5%A4%A9%E4%B8%8B%E3%80%8B%E7%B8%A3%E5%B8%82%E6%95%99%E8%82%B2%E5%8A%9B%E8%AA%BF%E6%9F%A5%EF%BC%9A%E6%96%B0%E8%AA%B2%E7%B6%B1%E4%B8%8A%E8%B7%AF%E5%89%8D%E5%A4%95%EF%BC%8C%E8%A1%8C%E6%94%BF%E6%B8%9B%E9%87%8F%E3%80%81%E8%AA%B2%E7%B6%B1%E6%BA%96%E5%82%99++%E9%82%84%E9%9C%80%E5%8A%AA%E5%8A%9B/【Chen, Y.-H., & Wang, Y.-L. (2018). 2018 key findings for education quality of administrative divisions. Retrieved from https://www.parenting.com.tw/article/5077684-2018%E3%80%8A%E8%A6%AA%E5%AD%90%E5%A4%A9%E4%B8%8B%E3%80%8B%E7%B8%A3%E5%B8%82%E6%95%99%E8%82%B2%E5%8A% 9B%E8%AA%BF%E6%9F%A5%EF%BC%9A%E6%96%B0%E8%AA%B2%E7%B6%B1%E4%B8%8A%E8%B7%AF%E5%89%8D%E5%A4%95%EF%BC%8C%E8%A1%8C%E6%94%BF%E6%B8%9B%E9%87%8F%E3%80%81%E8%AA%B2%E7%B6%B1%E6%BA%96%E5%82%99++%E9%82%84%E9%9C%80%E5%8A%AA%E5%8A%9B/】
- 臺灣PISA國家研究中心（2000）。閱讀樣本試題（中文版）。臺南市：國立臺南大學。【PISA in Taiwan. (2000). PISA released items-reading. Tainan, Taiwan: National University of Tainan.】
- 劉宜芳、柯華葳（2017）。線上閱讀研究之回顧與展望。教育科學研究期刊，62（2），61-87。doi:10.6209/JORIES.2017.62(2).03【Liu, I.-F., & Ko, H.-W. (2017). Online reading research: A selective review. Journal of Research in Education Sciences, 62(2), 61-87. doi:10.6209/JORIES.2017.62(2).03】
- 蘇慧珍、楊凱琳、陳佳陽（2017）。閱讀策略教學對高二學生數學學習表現的影響。教育科學研究期刊，62（1），33-58。doi:10.6209/JORIES.2017.62(1).02【Su, H.-C., Yang, K.-L., & Chen, C.-Y. (2017). Effects of teaching reading strategies on senior high school student’s mathematics performance. Journal of Research in Education Sciences, 62(1), 33-58. doi:10.6209/JORIES.2017. 62(1).02】
- 蔡曉楓（2017）。十二年國民基本教育國語文素養導向教學課程論述與設計理念。教育研究月刊，275，99-113。doi:10.3966/168063602017030275007【Tsai, H.-F. (2017). Preliminary discussion on the discourses of curriculum designs of the literacy-based Mandarin education in 12-year basic education curricula. Journal of Education Research, 275, 99-113. doi:10.3966/ 168063602017030275007】
- Bogdan, R. C., & Biklen, S. K.（2001）。質性教育研究：理論與方法（黃光雄、李奉儒、高淑清、鄭瑞隆、林麗菊、吳芝儀…蔡清田，譯）。嘉義市：濤石文化。（原著出版於1998年）【Bogdan, R. C., & Biklen, S. K. (2001). Qualitative research for education: An introduction to theory and methods (K.-H. Huang, F.-J. Lee, S.-C. Kao, J.-L. Cheng, L.-C. Lin, C.-Y. Wu, …C.-T. Tsai, Trans.). Chiayi, Taiwan: WaterStone. (Original work published 1998)】
- Arias, I. J. (2007). Selecting reading materials wisely. LETRAS, 1(41), 131-151.
- Bond, T. G., & Fox, C. M. (2015). Applying the Rasch model: Fundamental measurement in the human sciences (3rd ed.). New York, NY: Routledge.
- Chall, J. S. (1983). Stages of reading development. New York, NY: McGraw-Hill.
- Chall, J. S. (1996). Stages of reading development (2nd ed.). Fort Worth, TX: Harcourt Brace College.
- DeVellis, R. F. (2017). Scale development: Theory and applications (4th ed.). Thousand Oaks, CA: Sage.
- Dorans, N. J., & Puhan, G. (2017). Contributions to score linking theory and practice. In R. E. Bennett & M. von Davier (Eds.), Advancing human assessment: The methodological, psychological and policy contributions of ETS (pp. 79-132). Princeton, NJ: Educational Testing Service.
- Ebel, R. L. (1954). Procedures for the analysis of classroom tests. Educational and Psychological Measurement, 14(2), 352-364. doi:10.1177/001316445401400215
- Florida Department of Education. (2010). FACT 2.0 reading test item specifications grades 9-10. Tallahassee, FL: Author.
- Garrod, S., & Sanford, T. (2012). Language in action. In N. Braisby & A. Gellatly (Eds.), Cognitive psychology (pp. 197-223). Oxford, UK: Oxford University Press.
- Haladyna, T. M. (2004). Developing and validating multiple-choice test items (3rd ed.). Mahwah, NJ: Lawrence Erlbaum Associates.
- Haladyna, T. M., & Rodriguez, M. C. (2013). Developing and validating test items. New York, NY: Routledge.
- Holland, P. W., & Wainer, H. (1993). Differential item functioning. New York, NY: Routledge.
- Kintsch, W. (1988). The role of knowledge in discourse comprehension: A construction-integration model. Psychotocical Review, 95(2), 163-182. doi:10.1016/S0166-4115(08)61551-4
- Kintsch, W. (2013). Revisiting the construction-integration model of text comprehension and its implications for instruction. In D. E. Alvermann, N. J. Unrau, & R. B. Ruddell (Eds.), Theoretical models and processes of reading (6th ed., pp. 807-839). Newark, DE: International Reading Association.
- Mullis, I. V. S., Martin, M. O., Foy, P., & Hooper, M. (2017). ePRILS 2016 international results in online informational reading. Chestnut Hill, MA: International Association for the Evaluation of Educational Achievement.
- Organisation for Economic Co-operation and Development. (2012). PISA 2009 technical report. Paris, France: Author.
- Organisation for Economic Co-operation and Development. (2014). Competency framework. Paris, France: Author.
- Organisation for Economic Co-operation and Development. (2015). PISA 2018 stimulus submission guidelines: Reading literacy. Paris, France: Author.
- Organisation for Economic Co-operation and Development. (2016). PISA 2018 draft analytical frameworks. Paris, France: Author.
- Organisation for Economic Co-operation and Development. (2017). PISA 2015 assessment and analytical framework: Science, reading, mathematic, financial literacy and collaborative problem solving (Rev. ed.). Paris, France: Author.
- Organisation for Economic Co-operation and Development. (2018a). PISA 2018 reading literacy framework. Paris, France: Author.
- Organisation for Economic Co-operation and Development. (2018b). Preparing our youth for an inclusive and sustainable world: The OECD PISA global competence framework. Paris, France: Author.
- Pitts, M. M. (1983). Comprehension monitoring: Definition and practice. Journal of Reading, 26(6), 516-523.
- Salehi, M., & Tayebi, A. (2012). Differential item functioning: Implications for test validation. Journal of Language Teaching and Research, 3(1), 84-92. doi:10.4304/jltr.3.1.84-92
- Schwanenflugel, P. J., & Knapp, N. F. (2016). The psychology of reading: Theory and applications. New York, NY: Guilford.
- Varma, S. (2003). Preliminary item statistics using point-biserial correlation and p-values. Morgan Hill, CA: Educational Data Systems.
- Verhelst, N. (2004). Reference supplement to the manual for relating language examinations to the common european framework of reference for languages: Learning, teaching, assessment − Section C: Classical test theory. Strasbourg Cedex, France: Council of Europe.
- Wu, M., Tam, H. P., & Jen, T.-H. (2016). Educational measurement for applied researchers: Theory into practice. Singapore: Springer Nature.