A major barrier to the deployment of intelligent tutoring systems is the time and effort required to author the lesson materials. This presentation describes a new technology that helps authors of intelligent tutoring systems to quickly find the content they need for their lessons.
Context
A major barrier to the deployment of intelligent tutoring systems is the time and effort required to author the lesson materials. Even if a repository of relevant documents is available, time must be taken to find the snippets that can best serve as the basis for a lesson. Keyword searches typically used to search repositories often fail to match relevant material, produce many false positives, and return whole documents that must be searched again for the relevant snippets. This presentation describes emerging technology that drastically reduces the time and effort required to find tutoring content and thereby breaks down a significant barrier to the deployment of intelligent tutoring systems.
The US military is one of the largest educators in the world. Military personnel spend much of their time in training activities throughout their careers. The high cost and high importance of training has prompted the US Army to invest in intelligent tutoring technology that can provide instruction adapted to each learner’s needs (Sottilare, 2015). However, the benefits of intelligent tutoring systems are currently offset by the cost of authoring these systems. While the Central Army Registry, for example, provides a valuable repository of legacy instructional material, finding the snippets of information needed for a tutoring lesson is challenging because (1) keywords in the query may not match the technical terminology used in Army manuals, (2) keywords in the query often match unintended words in documents, and (3) the manuals retrieved by the search are often hundreds of pages long forcing authors to spend additional time searching for information within the manuals.
This presentation describes a new technology that helps authors of intelligent tutoring systems to quickly find the content they need. Content Retrieval and Extraction for Advanced Tutoring Environments (CREATE) is a research prototype that uses semantic search and document segmentation to help find just the snippets authors need for their lessons. Semantic search uses concept hierarchies and automatic deduction to match search concepts to the concepts that appear in documents. Document segmentation indexes individual paragraphs, diagrams, slides, and other snippets, so they can be matched and retrieved independently of the larger document.
Methods and Results
We conducted a double-blind, mixed-methods experiment to compare the efficiency of instructional systems designers using CREATE with a control group using traditional keyword search. In a blind comparison by a separate set of experienced instructional systems designers, the group using CREATE found five times more directly-relevant snippets than the control group, indicating that CREATE technology has a strong potential for reducing the cost of authoring intelligent tutoring systems.
Discussion
Of course the reduced content-retrieval times come at a price: both the effort required to build the concept hierarchies required for semantic search and the effort to implement document segmentation are significant costs. However, these are one-time payments—once the concept hierarchies and document segmentation are in place, they can support every subsequent search. In contrast, while keyword search has low up-front costs, the efficiency of every subsequent search suffers. The one-time investment in semantic search will therefore be highly cost-effective in a large educational organization.
Conclusions
While intelligent tutoring systems are an important innovation in instructional technology, the cost of authoring these systems is a significant barrier to realizing their potential. The CREATE system demonstrates technology that significantly lowers that barrier by drastically reducing the time and effort required to retrieve raw lesson content from document repositories.
References
Sottilare, R. (2015). Challenges in Moving Adaptive Training & Education from State-of-Art to State-of-Practice. In Proceedings of the “Developing a Generalized Intelligent Framework for Tutoring (GIFT): Informing Design through a Community of Practice” Workshop at the 17th International Conference on Artificial Intelligence in Education (AIED 2015), Madrid, Spain.