Journal directory listing - Volume 63 (2018) - Journal of Research in Education Sciences【63(4)】December (Special Issue: The Concepts and Practices for Achieving Literacy)
Directory
(Special Issue) Development of Assessments for Scientific Literacy Based on Curriculum Guidelines for 12-Year Basic-Education in Science Domains
Author: Pei-Ling Lin (Science Education Centre, National Taiwan Normal University), Chang-Chih Pan (Department of Earth Sciences,Research Center for Psychological and Educational Testing, National Taiwan Normal University), Shao-Zu Su (Department of Educational Psychology & Counseling, Research Center for Psychological and Educational Testing, National Taiwan Normal University), Po-Hsi Chen (Department of Educational Psychology & Counseling, Institute for Research Excellence in Learning Sciences, National Taiwan Normal University)
Vol.&No.：Vol. 63, No.4
Date：December 2018
Pages：295-337
DOI：10.6209/JORIES.201812_63(4).0010
Abstract：
The purpose of this research is to analyse the curriculum guidelines in science domains within the 12-year Basic Education and to explore and develop a reliable and valid assessment model for scientific literacy. This study is conducted in three parts: First, the definitions of scientific literacy that are established in the literature are classified and examined. Second, a coding list is established by decoding the implications of the curriculum guidelines in the science domains, and this decoded result is supported by examining the definitions of scientific literacy from the literature. A unified assessment framework is established for drafting scientific literacy-oriented exam questions. Third, the coding list and assessment framework are used to develop scientific literacy-oriented exam questions, and based on this framework, a meta-analysis of unqualified exam questions is conducted to revise these exam questions. The aforementioned coding list and assessment framework are used to develop and test the scientific literacy-oriented exam questions. The results indicates that most of the scientific literacy-oriented exam questions developed in this study have good quality and follow the curriculum guidelines of the 12-Year Basic Education in science domains. Regarding the unqualified exam questions, the assessment framework for scientific literacy developed in this study provides recommendations for revision, which can help draft and correct exam questions. In sum, the proposed assessment framework for scientific literacy can be used to further develop, modify, and classify various assessment tools and as a reference for future teachers and researchers.
Keywords：12-year basic-education, assessment for scientific literacy, scientific literacy
《Full Text》
References：
- 任宗浩（2007）。評量架構。載於張俊彥（主編），TIMSS 2007國際數學與科學教育成就趨勢調查（pp. 13-63）。臺北市：國立臺灣師範大學。 【Jen, T.-H, J. (2007). Assessment framework. In C.-Y. Chang (Ed.), TIMSS 2007 trends in international mathematics and science study (pp. 13-63). Taipei, Taiwan: National Taiwan Normal University.】
- 李文旗、張俊彥（2005）。中學生應達到的地球科學素養？─中學地科老師的觀點。師大學報：科學教育類，50（2），1-27。doi:10.6300/JNTNU.2005.50(2).01 【Lee, W.-C., & Chang, C.-Y. (2005). Taiwan’s secondary school teachers’ expectations with regard to the earth science literacy of their students. Journal of National Taiwan Normal University: Mathematics & Science Education, 50(2), 1-27. doi:10.6300/JNTNU.2005.50(2).01】
- 宋曜廷、周業太、曾芬蘭（2014）。十二年國民基本教育的入學考試與評量變革。教育科學研究期刊，59（1），1-32。doi:10.6209/JORIES.2014.59(1).01 【Sung, Y.-T., Chou, Y.-T., & Tseng, F.-L. (2014). Standards-based assessments for 12-year basic education in Taiwan. Journal of Research in Education Sciences, 59(1), 1-32. doi:10.6209/JORIES.2014.59(1).01】
- 林宜平、吳亭亭、黎雅如、周桂田、鄭尊仁（2010）。台灣成年民眾對奈米產品與科技的公眾感知。台灣公共衛生雜誌，29（5），431-439。doi:10.6288/TJPH2010-29-05-0 【Lin, Y.-P., Wu, T.-T., Li, Y.-R., Chou, K.-T., & Cheng, T.-J. (2010). Perceptions of nano-products and nanotechnology by Taiwanese adults. Taiwan Journal of Public Health, 29(5), 431-439. doi:10.6288/TJPH 2010-29-05-06】
- 施琮仁（2015）。不同媒體平台對公眾參與科學決策能力之影響：以奈米科技為例。新聞學研究，124，165-213 【Shih, T.-J. (2015). Media and public ability to participate in scientific decision-making: Using nanotechnology as a case study. Mass Communication Reserch, 124, 165-213.】
» More
- 任宗浩（2007）。評量架構。載於張俊彥（主編），TIMSS 2007國際數學與科學教育成就趨勢調查（pp. 13-63）。臺北市：國立臺灣師範大學。 【Jen, T.-H, J. (2007). Assessment framework. In C.-Y. Chang (Ed.), TIMSS 2007 trends in international mathematics and science study (pp. 13-63). Taipei, Taiwan: National Taiwan Normal University.】
- 李文旗、張俊彥（2005）。中學生應達到的地球科學素養？─中學地科老師的觀點。師大學報：科學教育類，50（2），1-27。doi:10.6300/JNTNU.2005.50(2).01 【Lee, W.-C., & Chang, C.-Y. (2005). Taiwan’s secondary school teachers’ expectations with regard to the earth science literacy of their students. Journal of National Taiwan Normal University: Mathematics & Science Education, 50(2), 1-27. doi:10.6300/JNTNU.2005.50(2).01】
- 宋曜廷、周業太、曾芬蘭（2014）。十二年國民基本教育的入學考試與評量變革。教育科學研究期刊，59（1），1-32。doi:10.6209/JORIES.2014.59(1).01 【Sung, Y.-T., Chou, Y.-T., & Tseng, F.-L. (2014). Standards-based assessments for 12-year basic education in Taiwan. Journal of Research in Education Sciences, 59(1), 1-32. doi:10.6209/JORIES.2014.59(1).01】
- 林宜平、吳亭亭、黎雅如、周桂田、鄭尊仁（2010）。台灣成年民眾對奈米產品與科技的公眾感知。台灣公共衛生雜誌，29（5），431-439。doi:10.6288/TJPH2010-29-05-0 【Lin, Y.-P., Wu, T.-T., Li, Y.-R., Chou, K.-T., & Cheng, T.-J. (2010). Perceptions of nano-products and nanotechnology by Taiwanese adults. Taiwan Journal of Public Health, 29(5), 431-439. doi:10.6288/TJPH 2010-29-05-06】
- 施琮仁（2015）。不同媒體平台對公眾參與科學決策能力之影響：以奈米科技為例。新聞學研究，124，165-213。 【Shih, T.-J. (2015). Media and public ability to participate in scientific decision-making: Using nanotechnology as a case study. Mass Communication Reserch, 124, 165-213.】
- 張欣怡、張淑苑、羅慶璋、洪振方（2015）。知識整合數位課程促進學生科學素養：以化學反應概念為例。教育科學研究期刊，60（3），153-181。doi:10.6209/JORIES.2015.60(3).06 【Chang, H.-Y., Chang, S.-Y., Lo, C.-J., & Hung, J.-F. (2015). Using a knowledge-integration-based digital curriculum to facilitate scientific literacy in learning chemical reactions. Journal of Research in Education Sciences, 60(3), 153-181. doi:10.6209/JORIES.2015.60(3).06】
- 教育部（2014）。十二年國民基本教育課程綱要─總綱。臺北市：作者。 【Ministry of Education. (2014). Twelve years of national basic education curriculum outline in general. Taipei, Taiwan: Author.】
- 教育部（2016）。十二年國民基本教育課程綱要（國民中小學暨普通型高級中等學校）─自然科學領域（草案）。臺北市：作者。 【Ministry of Education. (2016). The draft of twelve years of national basic education curriculum outline (national primary and secondary schools and ordinary high school ) − Natural sciences. Taipei, Taiwan: Author.】
- 教育部、國立臺灣師範大學（主編）（2006）。九年一貫課程自然與生活科技學習領域─科學素養的內涵與解析（三版）。臺北市：作者。 【Ministry of Education, & National Taiwan Normal University. (Eds.). (2006). Grade 1-9 curriculum “science and technology” − The connotation and analysis of scientific literacy (3rd ed.). Taipei, Taiwan: Author.】
- 國立臺灣師範大學心理與教育測驗研究發展中心（2015）。106年國中教育會考問與答。臺北市：作者。 【The Research Center for Psychological and Educational Testing, National Taiwan Normal University. (2015). Q&A for the standard-based assessment of student achievement. Taipei, Taiwan: Author.】
- 國立臺灣師範大學心理與教育測驗研究發展中心（2016）。國民中學學生學習成就評量標準（試行版）。取自https://www.sbasa.ntnu.edu.tw/SBASA/Assessment/sbasa.pdf 【The Research Center for Psychological and Educational Testing, National Taiwan Normal University. (2016). The standard of standard-based assessment of student achievement for elementary and junior school students (Trial version). Retrieved from https://www.sbasa.ntnu.edu.tw/SBASA/Assessment/sbasa.pdf】
- 國家教育研究院（2018）。素養導向「紙筆測驗」要素與範例試題（定稿版）。臺北市：作者。 【National Academy for Educational Research. (2018). The elements and samples for literacy-oriented “paper and pencil test” (Final version). Taipei, Taiwan: Author.】
- 靳知勤（2002）。效化「基本科學素養」問卷。科學教育學刊，10（3），287-308。doi:10.6173/ CJSE.2002.1003.04 【Chin, C.-C. (2002). The validation of the test of basic scientific literacy for the use in Taiwan. Chinese Journal of Science Education, 10(3), 287-308. doi:10.6173/CJSE.2002.1003.04】
- 靳知勤（2007）。科學教育應如何提升學生的科學素養──臺灣學術精英的看法。科學教育學刊，15（6），627-646。doi:10.6173/CJSE.21007.1506.02 【Chin, C.-C. (2007). A reflection on the science education of Taiwan − The voice from the elites in Taiwan. Chinese Journal of Science Education, 15(6), 627-646. doi:10.6173/CJSE.21007.1506.02】
- 臺灣2015 PISA國家研究中心（2016）。2015學生能力國際排名。取自http://pisa2015.nctu. edu.tw/pisa/index.php/tw/rank/41-2015-rank 【Taiwan 2015 PISA National Research Center. (2016). 2015 International ranking of PISA students’ ability. Retrieved from http://pisa2015.nctu.edu.tw/pisa/index.php/tw/rank/41-2015-rank】
- 陳冠利、劉湘瑤、陳柏熹、黃書涵（2015）。以情境式試題評量中學生能源素養。教育科學研究期刊，60（2），167-196。doi:10.6209/JORIES.2015.60(2).06 【Chen, K.-L., Liu, S.-Y., Chen, P.-H., & Huang, S.-H. (2015). Using contextualized assessment to measure the energy literacy of middle and high school students. Journal of Research in Education Sciences, 60(2), 167-196. doi:10.6209/JORIES.2015.60(2).06】
- 陳柏熹、黃馨瑩、陳郁欣、葉泰廷、蘇少祖（2015）。大學生基本素養測驗的發展及信度效度分析。教育科學研究期刊，60（3），95-126。doi:10.6209/JORIES.2015.60(3).04 【Chen, P.-H., Huang, H.-Y., Chen, Y.-H., Yeh, T.-T., & Su, S.-T. (2015). Evaluation of the psychometric properties of the general literacy test for university students. Journal of Research in Education Sciences, 60(3), 95-126. doi:10.6209/JORIES.2015.60(3).04】
- 陳柏熹、宋曜廷（2017，11月）。十二年國教新課綱素養評量實踐：從定義到評量配套。論文發表於12年國教新課綱素養評量的實踐研討會，臺北市。 【Chen, P.-H., & Sung, Y.-T. (2017, November). The practice of evaluation of 12-year compulsory education: From definition to evaluation. Paper presented at the Practice of Evaluation of 12-Year Compulsory Education Conference, Taipei, Taiwan.】
- American Association for the Advancement of Science. (1989). Project 2061. Science for all American. New York, NY: Oxford University Press.
- Allum, N. (2010). Science literacy. In S. H. Priest (Ed.), Encyclopedia of science and technology communication (pp. 724-727). Thousand Oaks, CA: Sage.
- Bauer, M. W. (2009). The evolution of public understanding of science − Discourse and comparative evidence. Science, Technology and Society, 14(2), 221-240. doi:10.1177/097172180901400202
- Beck, U. (2012). Risk society: Towards a new modernity (M. Ritter, Trans.). London, UK: Sage. (Original work published 1992)
- Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956). Taxonomy of educational objectives: The classification of educational goals, handbook 1: Cognitive domain. New York, NY: David McKay.
- Brossard, D., & Shanahan, J. (2006). Do they know what they read? Building a scientific literacy measurement instrument based on science media coverage. Science Communication, 28(1), 47-63. doi:10.1177/1075547006291345
- Bucchi, M., & Saracino, B. (2016). “Visual science literacy”: Images and public understanding of science in the digital age. Science Communication, 38(6), 812-819. doi:10.1177/10755470166 77833
- Bybee, R. W. (1997). Toward an understanding of scientific literacy. In W. Graber & C. Bolte (Eds.), Scientific literacy (pp. 37-68). Kiel, Germany: IPN.
- Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37-46. doi:10.1177/001316446002000104
- de Ayala, R. J. (2009). The theory and practice of item response theory. New York, NY: Guilford.
- Deboer, G. E. (2000). Scientific literacy: Another look at its historical and contemporary meanings and its relationship to science education reform. Journal of Research in Science Teaching, 37(6), 582-601. doi:10.1002/1098-2736(200008)37:6<582::AID-TEA5>3.0.CO;2-L
- Durant, J. R., Evans, G. A., & Thomas, G. P. (1989). The public understanding of science. Nature, 340(6228), 11-14. doi:10.1038/340011a0
- Eivers, E., & Kennedy, D. (2006). The PISA assessment of scientific literacy. The Irish Journal of Education, 37, 101-119.
- Eshach, H., Hwang, F.-K., Wu, H.-K., & Hsu, Y.-S. (2013). Introducing Taiwanese undergraduate students to the nature of science through Nobel Prize stories. Physical Review Special Topics − Physics Education Research, 9(1), 010116. doi:10.1103/PhysRevSTPER.9.010116
- Fang, S.-C., Hsu, Y.-S., Chang, H.-Y., Chang, W.-H., Wu, H.-K., & Chen, C.-M. (2016). Investigating the effects of structured and guided inquiry on students’ development of conceptual knowledge and inquiry abilities: A case study in Taiwan. International Journal of Science Education, 38(12), 1945-1971. doi:10.1080/09500693.2016.1220688
- Hsin, M.-C., Chien, S., Hsu, Y.-S., Lin, C.-Y., & Yore, L.-D. (2016). Development and validation of a Taiwanese communication progression in science education. International Journal of Science and Mathematics Education, 14(1), 125-143. doi:10.1007/slo63-014-9589-y
- Hsu, Y.-S., Chang, H.-Y., Fang, S.-C., & Wu, H.-K. (2014). Developing technology-infused inquiry learning modules to promote science learning in Taiwan. In M. S. Khine (Ed.), Science education in East Asia pedagogical innovations and research-informed practices (pp. 373-403). Cham, Switzerland: Springer. doi:10.1007/978-3-319-16390-1_15
- Huang, H.-Y., Wang, W.-C., Chen, P.-H., & Su, C.-M. (2013). Higher-order item response models for hierarchical latent traits. Applied Psychological Measurement, 37(8), 619-637. doi:10.1177/ 0146621613488819
- Hurd, P. D. (1958). Science literacy: Its meaning for American schools. Educational Leadership, 16(1), 13-16.
- Jones, L. R., Wheeler, G., & Centurino, V. A. S. (2015). TIMSS 2015 − science framework. In I. V. S. Mullis & M. O. Martin (Eds.), TIMSS 2015 assessment frameworks (pp. 29-58). Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College.
- Kolstø, S. D. (2001). Scientific literacy for citizenship: Tools for dealing with the science dimension of controversial socioscientific issues. Science Education, 85(3), 291-310. doi:10.1002/sce.1011
- Kuo, C.-Y., Wu, H.-K., Jen, T.-H., & Hsu, Y.-S. (2015). Development and validation of a multimedia-based assessment of scientific inquiry abilities. International Journal of Science Education, 37(14), 2326-2357. doi:10.1080/09500693.2015.1078521
- Laugksch, R. C., & Spargo, P. E. (1996). Construction of a paper-and-pencil test of basic scientific literacy based on selected literacy goals recommended by the american association for the advancement of science. Public Understanding of Science, 5(4), 331-359. doi:10.1088/0963- 6625/5/4/003
- Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159-174. doi:10.2307/2529310
- Linacre, J. M., & Wright, B. D. (1994). Reasonable mean-square fit values. Rasch Measurement Transactions, 8(3), 370.
- Lin, H., Hong, Z.-R, & Huang, T.-C. (2012). The role of emotional factors in building public scientific literacy and engagement with science. International Journal of Science Education, 34(1), 25-42. doi:10.1080/09500693.2010.551430
- Lin, P.-L., Tseng, Y.-H., Liu, H.-H., & Chang, C.-Y. (2012, April). What the media wants us to believe: Investigating Taiwan media attitudes towards nanotechnology. Paper presented at 12th International Public Communication of Science and Technology Conference, Florence, Italy.
- Miller, J. D. (1983). Scientific literacy: A conceptual and empirical review. Daedalus, 112(2), 29-48.
- Miller, J. D. (1998). The measurement of civic scientific literacy. Public Understanding of Science, 7(3), 203-223. doi:10.1088/0963-6625/7/3/001
- National Research Council. (1996). National science education standards. Washington, DC: National Academy Press. doi:10.17226/4962
- Organisation for Economic Co-operation and Development (2013). PISA 2015 draft science framework. Retreived from http://www.oecd.org/pisa/pisaproducts/Draft%20PISA%202015% 20Science%20Framework%20.pdf
- Organisation for Economic Co-operation and Development. (2012). PISA 2015 item submission guidelines: Scientific literacy. Retreived from https://www.oecd.org/pisa/pisaproducts/Submission- Guidelines-Science.pdf
- Peters, H. P., Brossard, D., de Cheveigné, S., Dunwoody, S., Kallfass, M., Miller, S., & Tsuchida, S. (2008). Science-media interface: It’s time to reconsider. Science Communication, 30(2), 266-276. doi:10.1177/1075547008324809
- Pidgeon, N., & Fischhoff, B. (2011). The role of social and decision sciences in communicating uncertain climate risks. Nature Climate Change, 1(1), 35-41. doi:10.1038/nclimate1080
- Rasch, G. (1960). Studies in mathematical psychology: I. probabilistic models for some intelligence and attainment tests. Copenhagen, Denmark: Danish Institute for Educational Research.
- Rundgren, C.-J., Rundgren, S.-N. C., Tseng, Y.-H., Lin, P.-L., & Chang, C.-Y. (2010). Are you SLiM? Developing an instrument for civic scientific literacy measurement (SLiM) based on media coverage. Public Understanding of Science, 21(6), 759-773. doi:10.1177/0963662510 377562
- Shamos, M. H. (1995). The myth of scientific literacy. New Brunswick, NJ: Rutgers University Press.
- Shih, T.-J., & Lin, C.-Y. (2017). Developing communication strategies for mitigating actions against global warming: Linking framing and a dual processing model. Environmental Communication, 11(6), 840-858. doi:10.1080/17524032.2016.1154886
- Stewart, C. O., McConnell, J. R., & Dickerson, D. L. (2017). Socioscientific and epistemic dimensions of support for science: Associations with science education and religiosity. International Journal of Science Education, Part B, 7(1), 1-13. doi:10.1080/21548455.2016. 1157644
- Sturgis, P., & Allum, N. (2004). Science in society: Re-evaluating the deficit model of public attitudes. Public Understanding of Science, 13(1), 55-74. doi:10.1177/0963662504042690
- Wu, H.-K., Kuo, C.-Y., Jen, T.-H., & Hsu, Y.-S. (2015). What makes an item more difficult? Effects of modality and type of visual information in a computer-based assessment of scientific inquiry abilities. Computers & Education, 85, 35-48. doi:10.1016/j.compedu.2015.01.007
- Wu, H.-L., Weng, H.-L., & She, H.-C. (2016). Effects of scaffolds and scientific reasoning ability on web-based scientific inquiry. International Journal of Contemporary Educational Research, 3(1), 12-24.
- Yang, F.-Y., Chen, Y.-H., & Tsai, M.-J. (2013). How university students evaluate online information about a socio-scientific issue and the relationship with their epistemic beliefs. Educational Technology & Society, 16(3), 385-399.
- Yang, F.-Y., Chang, C.-C., Chen, L.-L., & Chen, Y.-C. (2016). Exploring learners’ beliefs about science reading and scientific epistemic beliefs, and their relations with science text understanding. International Journal of Science Education, 38(10), 1591-1606. doi:10.1080/ 09500693.2016.1200763
- Yang, F.-Y., Liu, S.-Y., Hsu, C.-Y., Chiou, G.-L., Wu, H.-K., Wu, Y.-T., …Tsai, C.-C. (2018). High-school students’ epistemic knowledge of science and its relation to learner factors in science learning. Research in Science Education, 48(2), 325-344. doi:10.1007/s11165-016-9570-6
- Yang, W.-T., Lin, Y.-R., She, H.-C., & Huang, K.-Y. (2015). The effects of prior-knowledge and online learning approaches on students’ inquiry and argumentation abilities. International Journal of Science Education, 37(10), 1564-1589. doi:10.1080/09500693.2015.1045957