Building a Learning Management System that integrates Large Language Models and Retrieval-Augmented Generation (RAG)

Large Language Models (LLMs) have become a prominent entity in the last few years in our everyday life but especially in the academic world. Most times, they are used primarily by students to solve assignments in settings like exam preparation by using it to generate questionnaires, flashcards or to provide feedback on alternative solutions to solving exam questions. Although it comes with one major caveat: hallucinations and errors.

The vast majority of LLM platforms offer solutions targeted towards students whilst leaving educators with minimal support and regular Learning Management Systems (LMS) like Canvas or Open eClass often lack even basic generative functions. This creates a considerable asymmetry leaving educators overwhelmed due to the need to manage and teach course material, time-intensive research as well as administration of their department. Retrieval-augmented generation (RAG) has been used by LLM platforms to lower hallucination rates but LMS platforms haven’t incorporated such technology to help educators with their workflows.

This thesis will address the asymmetry by covering the design and building of a prototype platform focused on both educators and students alike, providing students with accurate preparation material aggregated by their educators as well as providing said educators with automations that involve LLMs to make course focused tasks easier to manage.

This research aims to create a platform that will demonstrate key capabilities like:

• RAG-based content generation from ingested course materials provided to the system by the educator
• Quiz and true/false question generation, automating assessment content creation based on course materials using templated prompts to ensure questions are relevant with variable difficulty and count
• Flashcard generation based on extracted questions and answers from the documents uploaded

Given the constraints of this thesis and the platform’s prototype status, evaluation will be done through technical benchmarks and an informal qualitative pilot with a small group of volunteer participants to validate the core concept.

The thesis will be structured in three phases. Phase one covers the background and literature review on LLMs, RAG ingestion and automated assessment design. Phase two will document the design and implementation of the platform covering the RAG pipeline, quiz and true/false generation as well as the flashcard generation. Phase three will present the evaluation results and conclude with a discussion of limitations and future work.