LearningClues Revolutionizes Education in College Lectures - Climate and Space Sciences and Engineering

One technological application is revolutionizing the way college students absorb lectures, after an interdisciplinary effort led to the invention of LearningClues. The brainchild of Professor Perry Samson, Ph.D., the application combines A.I. with natural language processing to deliver study guides on-demand.

LearningClues supports students seeking help with their studies by providing easily searchable lecture videos and contextual links to topics discussed in class. When instructors add LearningClues in Canvas the application is capable of mining class videos to identify key terms to produce a single, searchable database. That database which is available only to the instructors and students in the course can be tapped anytime, whether a student is studying for an exam or applying a concept they learned to the real world.

SPECIAL EVENT: Attend The Perry Samson Colloquium

Learning from Machine Learning

“In surveys of students we’ve learned that study guides are offered in only about 15% of courses while about 85% of students would welcome study guides to help focus their study,” said Samson, an Arthur Thurnau Professor in the U-M Department of Climate and Space Sciences and Engineering. “The question of ‘what should I study?’ is not uncommon, particularly in the first year or two of college. LearningClues analyzes what’s being said in class and/or visually presented in class. We are then able to use artificial intelligence to identify what we believe are the important words being discussed, and we index that in all these courses.”

The indexed terms stay in the database, so that students can search in their courses to find the exact moment in class videos when a concept was introduced. A video clip pops up and delivers the information directly to the student. If the instructor wishes, LearningClues can also mine the documents and links they’ve included in their Canvas site and link students to those resources automatically when topics discussed in class are mentioned in those documents or links. LearningClues can also point students to the appropriate pages in their textbook for a particular keyword or concept.

“It becomes a way for students to find information more efficiently,” said Samson. “Students are often challenged by competing demands on their time and welcome ways to be more efficient in their studies.”

The application goes beyond providing this invaluable database. It’s also programmed to create study guides automatically, with no human direction required.

“Since we are actively indexing what’s being said in class, we can then collect which words were discussed on any given day or time period,” said Samson. “In our new study guide application the words discussed over a period of the student’s selection are aggregated producing an automatically generated set of flashcards.”

Example illustrating a search for when “Coriolis force” was discussed in class videos. LearningClues shows in which lectures the term was discussed and at what specific moments in those lectures.

Reverse-Engineering the College Course

The LearningClues application was provided in over 400 courses in the University of Michigan’s College of Engineering during the winter semester, and it will be released in the Master of Applied Data Science program in 2023. In the long term, the vision is to make the technology available to colleges and universities everywhere. It is available now at the University of Michigan and will be released to several universities across the U.S. in the coming months.

“What we’re attempting is to reverse-engineer the learning objectives of the course using AI,” said Samson. “Based on what’s being discussed in a course we estimate what discipline it represents and identify terms and concepts that are commonly associated with that discipline. Instructors can, of course, overrule or edit the list of AI chosen key terms if they wish but it’s our goal to support students whether instructors choose to actively participate or not. ”

The technology is designed to adapt and learn with students as they learn.

“That’s the cool part,” said Samson. “Sudents are presented with suggested resources relevate to what was discussed in class and they will be able to provide feedback to us whether the resource was helpful or not. That informs our machine learning system, so that as more students use the system, the system gets smarter about which resources to suggest.”

A Powerhouse of Partnership

With funding from the National Science Foundation and support from U-M College of Engineering and Innovation Partnerships, many faculty and students have worked together to make this tech a reality. The application is based in research and built upon earlier technologies, expanding what’s possible in college lectures.

“When we built this thing, we built in some very strong safeguards, so that the only people who can search a course are students who took the course and the instructor,” said Samson.

Samson worked with a team of 15 students over the summer to release the application to new courses this fall. Students contributed to back-end and front-end web development, coming from programs in computer science, engineering, business, and architecture. Development will continue as a part of the Multidisciplinary Design Program in the College of Engineering.

“I am blessed with an enthusiastic team of students,” said Samson. “Obviously it affects their own learning. We’ve got students that are both in the School of Information and the College of Engineering working together to make the model smarter.”

He also partnered with ITS, CAEN, and the U-M School of Information to bring the vision to life.

“It literally takes a village to do this, and part of that village is people who are knowledgeable in what’s called natural language processing,” said Samson.

Associate Professor of Information and Computer Science Kevyn Collins Thompson, Ph.D., has been collaborating closely with Samson to provide expertise in natural language processing and search engine technology. In his work, he blends information retrieval, machine learning, natural language processing, and large-scale data mining to optimally connect people with information, especially helping them to learn and discover.

“On the technical side, this project has provided a beautiful opportunity to bring together several of my research themes, including search engines that help people learn, and reliable ways to predict the difficulty of STEM content for individual students,” he said. “On the personal side, working with Perry has been truly inspiring: every day he brings relentless energy to pursuing innovative ideas, and a true passion for helping students.”

The technology could also be leveraged for other purposes. Provided to instructors, it could show what concepts have been learned in previous courses. Provided to department leadership, it could demonstrate what areas of curriculum are redundant across an academic program. Provided to alumni, it could make a college career available at the graduate’s fingertips.

“You would have the entire trajectory of your educational experience available to you through search, with the potential that even alumni could go back and hear the lecture they should have heard years ago,” said Samson.

If you are an instructor or GSI interested in exploring LearningClues in a course, please contact Perry Samson at [email protected].