Skillbox and HSE: A model for predicting adult engagement in learning

Main article: Data mining Data mining

2023: Model development showing the engagement of online platform users

On August 30, 2023, the company Skillbox announced that education HSE it had developed a model with the team of the International Practice Assessment and Innovation Laboratory that predicts the involvement of users of the online platform. It takes into account the number of classes started, homework completed, results, testings the number of overlooked videos, passing results tests and other characteristics. The value of the model in the possibilities of further practical application timely is the development of measures to support students with low involvement and help in achieving their educational goals. The company plans information to use student engagement to build personalized training and support tracks.

One of the factors of effective education of adults is their involvement in the educational process. The result in this case refers to the achievement of the goals that the students set themselves. At the same time, there is no automated system for measuring engagement, which also takes into account the personal experience of students, in the education market. Thus, the goals of the research project were: scientific proof that engagement can be measured through digital traces, as well as building a model that will help track the degree of involvement of students and develop systemic measures to support them.

Skillbox in its model of educational product has always been focused on busy adults who consciously choose asynchronous learning and independently adjust it to their life rhythm. We, however, understand that learning engagement with this approach may be falling due to learning interruptions, and actively investigate user behavior to build a more impactful student-platform relationship. commented Natalia Vlodavskaya, Service Director, Skillbox.

Work on the model was carried out in four stages:

1. Data collection. This allowed us to determine involvement by more traditional methods, and then compare these results with the results of the developed model. The questionnaire consisted of 13 statements spread across three components of engagement (according to Jennifer Fredricks' classic engagement model) - behavioural (e.g., "I regularly do homework"), cognitive ("If I don't understand something, I try to figure it out to the end"), and emotional ("I rarely feel helpless while teaching the course"). The final sample was 2,234 users.

2. Audience segmentation. According to the survey results, three segments of students were identified by level of involvement: low, medium and high.

3. Building a predictive model. The model was developed using algorithmic machine learning. The number of classes started, homework completed, the number of watched videos, the sum of all attempts to pass tests and other characteristics were taken into account.

4. Model validation. The final stage after obtaining the data and developing the model was an interview with students, with the help of which they checked how the conclusions obtained by analyzing digital traces correlate with their subjective experience.

In recent years, the research community's interest in building automated systems for monitoring learners' experiences and progress has naturally grown. And this task is not only and not so much technical - such systems allow us to better understand the factors associated with the involvement, motivation, well-being of students, look at them in dynamics, and, as a result, design educational experience more efficiently. said Yulia Gerasimova, project manager, Institute of Education, Higher School of Economics.

Measurement of engagement at all stages implies the possibility of using digital traces and their automatic detection.