Learning analytics

In line with the common UN capacity development approach and firmly anchored under the ILO capacity development strategy, capacity development is understood by ITCILO as a process through which individuals, organizations and societies obtain, strengthen and maintain the capabilities to set and achieve their own development objectives over time. Capacity development may involve technical capacities (in such fields as employment, social security and health) and functional capacities (such as leadership, financial management, partnership and relationship building) and it may be implemented in several ways. 

While training is a common factor, capacity development is best achieved through learning by doing, fostering relationships and partnerships, supporting research and knowledge-sharing, participation in communities of practice, South-South learning initiatives, on-the-job training, and other learning techniques that empower individuals and institutions to take charge of development challenges.

The ITCILO capacity development approach goes on distinguish three mutually reinforcing and interdependent levels of capacity development: individual, organizational, and resulting from the enabling environment. The first means increasing the skills and abilities of individuals; the second focuses on increasing the capacity of organizations to fulfil their mandates; and the third involves improvements in policies, legislation, regulations, labour market institutions, and societal systems.

The learning analytics advisory services offered by the Centre are targeted at the organizational level, here with focus on organizations linked to the ILO core constituency and other ILO development partners that offer learning services. 

Educational data science – or short Learning Analytics* cuts through almost all educational technology disciplines and is key to many other technological innovations for educational/training institutions. The introduction of a learning analytics system in an organization contributes to the overall goal of its digital transformation and strengthens its capacity to systematically analyse the data about its learners’ behaviour during the full learning cycle.

The scope and scale of the potential impact of using data to inform decision-making in training has significantly increased over the last few years. We are now at a stage where data can be automatically harvested at previously unimagined levels of granularity and variety. Analysis of this data has the potential to provide evidence-based insights into learner abilities and patterns of behaviour that in turn can provide crucial action points to guide course design, development of new training offers, improvement of outcomes for all learners, and thus a contribution to the economic and social well-being of the community.

There is currently a clear high demand for using learning data to improve the whole learning cycle. This spans from collecting and estimating the prior knowledge of learners about a certain topic, to the actual learning process and its assessment. The current shift to digital learning formats can be regarded as an advantageous opportunity to further develop the LA strategy and the digital transformation of training organizations.

*Learning Analytics is the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs. https://www.solaresearch.org/about/what-is-learning-analytics/ 

The ITCILO has a rich ecosystem of tools, services and a variety of data streams that are being utilized for prediction and analytics. Learning analytics can provide different levels of insights whether on the level of a single course thus addressing the needs of individuals (micro-level), on the level of a collection of courses thus providing information for course managers (meso-level) or on the level of a whole curriculum taking a bird view on a directory of courses and providing insights at the governance level by monitoring learning behaviour across courses and even across different disciplines (macro-level). Depending on which level the LA takes place, different objectives and information are of relevance and can be monitored.

In the ITCILO, the macro-level is closely connected to the strategy goals of the organisation and is reported according to core KPIs used for measuring the organizational performance. With LA, new metrics from data can be computed and reported next to already established KPIs. At the micro-level, the eCampus micro-level dashboard tracks courses and activities completion states as well as grades of learners, thus enabling course managers to closely monitor their courses and participants status. 

The ITCILO helps organizations answer the following guiding questions for establishing their LA concepts:

• What are the needs of your stakeholders for a LA solution and how can they be involved in the software development process?
• How can different stakeholder groups be supported by LA tools in a meaningful way? 
• How can shortcomings of a learning system be identified and innovated with LA?
• What are effective visualisations to support awareness and reflection processes of different stakeholder groups?
• What are promising predictors from a dataset to forecast the study success of learners?
• How to recommend or cluster learners into groups to recommend personalised information or learning paths to them?
• What kind of online learning tools are used in your organisation? 
• For which of those do you have access to the data? 
• How meaningful is this data in terms of your organisational KPIs on micro, meso, and macro levels?
• How can a value-based design in terms of ethics and privacy be considered in the software development process for LA solutions?
• How can the notion of ‘informed consent’ be supported by technology?
• How can you train your stakeholders to be more knowledgeable and responsible about/for their data?
• How can you train information and data literacy skills of your stakeholders?