QMUL has recently implemented a new Student Learning Engagement policy that outlines how we should use learner engagement analytics to improve our students’ experiences and educational outcomes. But what are learner engagement analytics, and how can we use them inclusively to support all of our students’ learning?
The Society for Learning Analytics Research (SoLAR) defines learning analytics as “the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimising learning and the environments in which it occurs.” This data might include QMPlus data, such as log-ins, activity completions, quiz scores and assessment submissions; attendance and QReview data; and academic performance data, such as grades and progression. It can also include student feedback and students’ demographic and background information.
As you can see, the data are abundant but it’s vital that we don’t lose sight of what this is really about: our students and their learning. The new policy places students firmly at the heart of our work with learner engagement analytics, advocating a supportive and constructive approach, rather than a punitive one. It lays out six principles, developed in consultation with staff and students, to guide our use of learner engagement analytics (LEA):
Learner engagement analytics can help us identify students who might be at risk and intervene early to provide support. They can also help us identify where we might need to adjust or improve our teaching approaches and learning materials. However, we need to think carefully about what the analytics data is actually telling us, question the assumptions we might make based on the data, and consider who might be disadvantaged or overlooked in the data.
Let’s look at the following example, of a hypothetical student called Amir:
Amir is studying for his MA in Medical Education online. He’s also an F2 in a busy hospital in the north of England, and he is married with a toddler. He chose QMUL’s MA because there is no requirement to attend live sessions – work and family commitments mean that he can’t guarantee he’d be able to attend regularly. However, Amir is very committed to his studies, setting aside time each week to work through the online learning materials and work on the assessments. He finds the materials really interesting, following links to the suggested readings and often doing further reading and research on topics of particular interest. He finds that it is easier to handwrite his reflections than to type them, as it makes it easier for him to think, and he has an app that he uses to convert these handwritten notes to typed notes if needed. Because of this, Amir rarely posts in discussion forums after the first week or so. He reads his peers’ posts though, and finds that he often refers to them in his own reflections. Amir was at Med School with one of the other students, so they often chat about what they’re learning in their regular WhatsApp chats or via email. In fact, when encouraged to share a draft lesson plan for peer feedback, Amir sends his draft to his friend via email, and she does the same.
The programme administrator conducts fortnightly reviews of engagement analytics in this module, using QMPlus log-ins, forum posting, webinar attendance and engagement with any assessment activities (including formative activities) to determine whether any students are at risk of not progressing. In her week 4 check, she sees that Amir has logged into QMPlus four times across the fortnight but hasn’t posted in any forums or attended the optional webinars. He also hasn’t shared a draft in the (optional) peer feedback activity. She decides to email him about his engagement.
As you can see from Amir’s example, the data doesn’t always give the full story! Therefore it is imperative that we adopt a critical approach to LEA usage and interpretation.
Some questions you might ask yourself when selecting engagement markers and thresholds for your module or programme and acting on the data include:
Within FMD, the implementation of the new policy is being led by Dr Jo Elliott (Reader, Learning Design in the Digital Education Studio) and George Borrie (Faculty Education Manager), in collaboration with the following Institute Learner Engagement leads:
BCI: Sergey Krysov and Pia CroninBlizard: Andrew Durham and Mic DessiDentistry: Dominic HurstIHSE: Ric Khine and Viki SoperWHRI: Sadani Cooray and Ade AdeleWIPH: Ava Kanyeredzi
Jo is also undertaking a QM Academy Fellowship this year, investigating opportunities for using learning analytics to support students and inform teaching in online learning environments.
Check out QM Academy’s resources on learner engagement analytics.
For more information about learner engagement analytics and their use in FMD, contact Jo, George or your institute’s Learner Engagement lead/s.