A Study on the Learning Process of Applying an Autotutoring System to Teach Reading Strategies to Elementary School Children
Author: Ju-Ling Chen (Research Center for Translation, Compilation and Language Education, National Academy for Educational Research), Hou-Chiang Tseng (Graduate Institute of Digital Learning and Education, National Taiwan University of Science and Technology), Chi-Shun Lien (Center for Teacher Education, National Chung Cheng University), Min-Ying Tsai (Research Center for Translation, Compilation and Language Education, National Academy for Educational Research), Yao-Ting Sung (Department of Educational Psychology and Counseling, National Taiwan Normal University)
Vol.&No.：Vol. 70, No. 2
Date：June 2025
Pages：195-233
DOI：https://doi.org/10.6209/JORIES.202506_70(2).0006
Abstract：
Purpose
This study has two main purposes. First, it aims to explore learners’ use of strategies and learning performance within an autotutor environment, using word embedding technique. Second, it investigates the learning processes and cognitive patterns of learners with different reading abilities. By analyzing these patterns, particularly the differences in learning trajectories and strategy application between groups, this study offers several suggestions for instructional design.
Literature Review
Many empirical studies support the idea that reading strategies can significantly enhance text comprehension and improve learning outcomes. However, existing research still has some limitations, particularly in addressing cognitive differences among readers and meeting the specific needs of learners with lower reading abilities. Interactive teaching methods, such as teacher-student dialogues, facilitate real-time communication and feedback, supporting active learning and enhancing performance, especially for students with lower reading abilities. However, these approaches require substantial human and time resources, making large-scale application challenging.
The autotutor system offers an effective solution to these limitations. By creating a technology-based learning environment, the autotutor system guides learners through interactive dialogue, encouraging them to engage in higher-order thinking, learn academic content, and apply strategic learning. Nevertheless, certain limitations exist in current autotutor research. First, researchers tend to rely on latent semantic analysis (LSA) techniques, which are less capable than neural networks in extracting semantic information. Second, most studies rely on general instructional guidelines to address subject knowledge and problem-solving but lack specific designs for incorporating reading strategies. Furthermore, few studies have focused on elementary school students as primary subjects.
Method
This study conducted a two-factor mixed design to examine the learning processes and outcomes of students receiving reading strategy instruction. The between-subject factor is reading ability (high vs. low), while the within-subject factor is test type (pre-test and post-test), with background knowledge as a covariate. Participants included 53 fifth-grade students with varying reading abilities from elementary schools in Taipei, New Taipei, and Keelung. Using the Automated Reading Elaboration System, students individually learned five reading strategies and relevant subject knowledge. Their learning processes, including learning time, learning paths, and cognitive patterns, were assessed along with their learning performance, which measured the frequency of reading strategy use and knowledge acquisition. To analyze learning performance, a two-factor mixed-design ANCOVA was employed, with reading ability as a between-subject factor, test type as a within-subject factor, background knowledge as a covariate, and the frequency of reading strategy use and knowledge content as dependent variables. Learning process analysis used a one-way ANOVA, treating reading ability as the independent variable, pre-test background knowledge as a covariate, and learning time and paths as dependent variables.
Results
The autotutor, developed using word embedding technology, facilitated learning by guiding students in applying reading strategies to understand academic content and provided an in-depth analysis of learning paths and cognitive patterns. Key findings from this study are as follows:
1. Strategy Use and Learning Performance Improvement. Both high- and low-reading-ability groups learned to apply reading strategies and showed improvement in learning performance, although the types of strategies used varied. For example, post-test scores in knowledge content were higher than pre-test scores across all students. Notably, the high-reading-ability group scored higher in content knowledge than the low-reading-ability group, suggesting that the autotutor can effectively improve learning outcomes. High-reading-ability students used background knowledge and organizational strategies, while low-reading-ability students relied more on connection inferences. Both groups demonstrated limited use of comprehension monitoring strategies, highlighting the need for further instruction in this area.
2. Differences in Learning Processes and Cognitive Patterns. Learning paths and cognitive patterns varied by reading ability. High-reading-ability learners showed shorter learning paths and required minimal assistance, while low-reading-ability learners needed more explicit prompts and sometimes overlooked the autotutor’s support role in learning. This finding emphasizes the importance of adaptive, differentiated designs in instructional materials and assessments to accommodate the diverse needs of learners.
Discussion
The study offers the following recommendations:
1. Enhancing Comprehension Monitoring. Comprehension monitoring was a less frequently used strategy among students, particularly for those with lower reading abilities. Verbal data indicated that, although students attempted to use comprehension monitoring to support understanding, the frequency was low. Future strategy instruction could incorporate demonstrations of self-assessment and error detection, supported by awareness scales that illustrate the effectiveness of comprehension monitoring. Additionally, demonstrating the impact of comprehension strategies through practical examples may improve student engagement with this technique.
2. Strengthening Elaborative Inference for Low-Reading-Ability Learners. Low-reading- ability learners often relied on shallow connection inferences, producing responses unrelated to content. The autotutor could support comprehension by providing background knowledge and guiding students in connecting the text to personal experiences, using explicit cues in instructional materials to aid understanding.
3. Personalized Instruction for High-Reading-Ability Learners. High-reading-ability learners displayed distinct learning processes and cognitive patterns, suggesting the benefit of personalized course designs. Data revealed that these students needed minimal guidance, while low-reading-ability students benefited from clear prompts. Conducting preliminary analyses of students’ cognitive patterns could inform individualized instructional materials and feedback mechanisms.
4. Using a Large Semantic Space to Enhance Strategy Learning. In contrast to previous work based on LSA, this study leverages a large-scale word2vec-based semantic space to enable precise meaning judgment, allowing for more accurate handling of learners’ open-ended responses and facilitating more realistic teaching within the autotutor.
Conclusion
In conclusion, this study highlights the effectiveness of autotutor systems in supporting learning, analyzing students’ learning paths, and identifying differences in strategy use. It provides insights for personalized reading instruction, showing that autotutors can effectively address diverse reading abilities. Future research could refine instructional design, assessment tools, and the ethical use of large language models to further enhance learning outcomes. For example, applying fine-tuning to integrate knowledge into Llama 3 could reduce incorrect information in responses and enable more sophisticated meaning analysis, allowing teaching strategies to be more flexible and diverse.
This study demonstrates that autotutor systems, combined with adaptive design and reading strategy instruction, effectively meet the needs of diverse learners, especially those with varying reading abilities. Future research can build on these findings to create more advanced autotutor environments for education.
Keywords：
natural language processing, representation learning, autotutor, word embedding, reading strategies