In line with the common UN capacity development approach and firmly anchored under the ILO capacity development strategy, capacity development is understood by ITCILO as a process through which individuals, organizations and societies obtain, strengthen and maintain the capabilities to set and achieve their own development objectives over time. Capacity development may involve technical capacities (in such fields as employment, social security and health) and functional capacities (such as leadership, governance, financial management, partnership and relationship building) and it may be implemented in several ways. While training is a common factor, capacity development can also be achieved through learning by doing, fostering relationships and partnerships, supporting research and knowledge-sharing, participation in communities of practice, South-South learning initiatives, on-the-job training, and other learning techniques that empower individuals and institutions to take charge of development challenges.

The ITCILO capacity development approach goes on distinguish three mutually reinforcing and interdependent levels of capacity development: individual, organizational, and resulting from the enabling environment. The first means increasing the skills and abilities of individuals; the second focuses on increasing the capacity of organizations to fulfil their mandates; and the third involves improvements in policies, legislation, regulations, labour market institutions, and societal systems.

The AI chatbot services offered by the Centre are targeted at the organizational level, here with a focus on organizations that desire to disseminate knowledge about and understand the internal governance framework or selected documents by stakeholders.

The AI chatbot harnesses Generative Artificial Intelligence, and more specifically Large Language Models (LLM) based on Generative Pre-Trained Transformer (GPT) technology in order to enable users to quickly access information and ask questions.

 Several advantages are offered by automatic language generation technology where a GPT is augmented with a database retrieval system for institutional knowledge management purposes.

• Firstly, the bot streamlines the process of accessing information, increasing efficiency by eliminating the need for extensive training or consultation with multiple sources.
• Secondly, the bot enhances accessibility through a user-friendly interface that can be accessed from any device with an internet connection 24/7.
• Lastly, in line with the current LLM technology, the bot improves answers’ consistency, by providing the sourced content and verifiable information to all staff members, reducing the risk of misunderstandings or misinterpretations as original documents can be easily checked.

One of the benefits of this type of chatbot is the ability to generate natural language responses, allowing users to query knowledge and interact with large amounts of internal documents more naturally. The bot does not uniquely rely on general information the Large Language Model was trained on, but a custom database of controlled documents is built to retrieve relevant information in ITCILO-specific governance documentation. This retrieval allows not only more accurate answers – having access to the documentary sources the bot’s answers were built on (i.e., the relevant PDF file)-, but it also yields a cyber-secure hosting for internal documentation. Moreover, the chatbot is not able to learn from its interactions with users, in order to avoid biases and make sure that the information processed comes only from the official documentation.

A closer look at the ITCILO's approach

The ITCILO AI-chatbot approach follows the PDCA cycle

PLAN: In the planning phase, the following questions are answered as part of the planning phase:

• Why are we doing this?  Identification of the main needs, and common inquiries
• What’s in scope? IT software, licenses, determination of documentation to be analyzed
• What is the expected outcome? Clarifying expectations
• How do we measure progress? Indicators and performance targets to monitor
• How are we going to do this? The how-to of building the AI chatbot
• When will the AI chatbot be ready and what are the costs of ownership? Implementation plans and activity budgets.

 A design sprint engages stakeholders to determine desired features and prototype guidelines.

DO: In the implementation phase, a project team develops a chatbot prototype tailored to frequent user queries. After internal testing and refining, a beta version is launched organization-wide, accompanied by comprehensive user training across various formats. The focus is on enabling effective system utilization.

CHECK: During the evaluation phase, the system's performance, accuracy, and user satisfaction are continuously monitored. Feedback from the users refines the system. Internal assessments alongside quantifiable performance indicators.

ACT: In the action phase, the pilot's outcomes are documented and shared with stakeholders. Based on the results, a decision is made on whether to roll out the Chatbot for broader use. This phase leverages insights gained from the pilot to inform wider strategic choices.

Example from the ITCILO

AnswerMate the AI chatbot in ITCILO

AnswerMate is a product of in-house collaboration, bolstered by technical guidance from the ILO research department. AnswerMater has access only to internal documents: Statute, Regulations, Rules, Circulars and information notes.

AnswerMate for ITCILO staff isn't just a chatbot; it's a knowledge companion, ready to empower the ITCILO staff with accurate, accessible, and consistent information whenever you need it.

AnswerMate offers advantages to two main groups: (a) all staff members and (b) the custodians of our internal documentation. Every staff member benefits from streamlined access to operational procedures. Some staff members also serve as custodians, responsible for developing and monitoring these processes post-release.

The internal governance documents were divided into text chunks and the text was vectorised. The chatbot's user interface, accessible through a homepage, was designed to include a question form and various admin features. To monitor and evaluate its performance, a database was implemented, recording questions, answers, and other interesting details. When the user submits a question to AnswerMate, a dynamic response page presents the results with interactive options.

ITCILO has developed a robust and unique methodology for testing and feature customization. In fact a long series of assessments was conducted to have better results from the different aspects of the retrieval and query mechanism. Among others: (1) the type of query, (2) the number of document chunks considered for the formulation of the answers, (3) the impact of the use of ChatGPT 3 or ChatGPT4, (4) the rank of the chunks by relevance with the question, (5) the impact of the addition of keywords and topics for each document chunks, and (6) the prompt of the user context.