QA Comm and Adv

 
Report brief
Quality Assuring the Centre's Communication and Advocacy Projects
1. Background

 

The Centre’s portfolio of communication and advocacy services expanded significantly during the 2020-21 biennium and has grown into a significant work stream of the Learning Innovation Programme in the Centre’s Training Department. In 2022, communication and advocacy activities continued to confirm their strategic importance to the Centre. Through a limited number of effective communication campaigns, the Centre was able to reach out to more than 103,000 people.

 

To consolidate and further grow its footprint in the market segment for communication and advocacy services in the coming years, the ITCILO has sought expert advice on aligning its channel-specific quality assurance system with international standards.

 

The document summarizes global good practice in quality-assuring communication projects, maps the Centre’s quality assurance practices against the standards for project management promoted by the International Standards Organization (ISO) and, where applicable, proposes concrete measures to narrow the performance gap.

 

A communication and advocacy service is in the following understood to be a project with a pre-established purpose, a multi-step implementation cycle with start and end date, with pre-defined outcomes and set deliverables tracked with key performance measures, with output-specific time-bound activities and with an activity budget.

2. Global Good Practice

 

The most extensive transitional study on strategic communication worldwide, the European Communication Monitor, has been conducted annually since 2007, with over 30,000 communication professionals taking part.

 

Additionally, several initiatives have been set in motion with the goal of developing best practices and appropriate standards.These include the dissemination and adoption of The Barcelona Principles (AMEC, 2010, 2015, 2022), as well as the founding of the Coalition for Public Relations Research Standards in 2011 and the Social Media Measurement Standards Conclave in 2012.

 

Moreover, an international Task Force on Standardization of Communication Planning and Evaluation Models has been established under the patronage of the IPR Measurement Commission with the aim to naming and clearly setting standards.

 

Only a few communication departments command advanced expertise in CommTech for converting internal workflows and communication activities to a digital format, and there is widespread concern regarding the lack of industry-wide quality standards in communication and advocacy.

 

A number of inter-governmental bodies have taken steps to explore and establish measures for quality assurance in communication and advocacy projects. For example, the European Commission has issued a Toolkit for the Evaluation of Communication Activities, while the Council of Europe has published guidelines to aid its project managers in effective communication across the entire project management cycle. 

 

In the private sector, AMEC has produced an integrated evaluation framework for planning and measuring their communications effectiveness. IDEO is a global company that has developed a widely used human-centred design toolkit.

 

Recurrent issues in quality assuring communication and advocacy projects are:

  • how to define the purpose and scope of a communication and advocacy project;
  • how to define a theory of change, and
  • how to measure performance.
3. Quality Assuring Communication and Advocacy projects at the ITCILO

 

The backbone of the Centre’s quality management system across its portfolio of capacity development services is the Plan-Do-Check-Act (PDCA) cycle promoted by the International Standards Organization (ISO) in its ISO 9001:2015(E) Standard for quality management systems.

 

PDCA cycle

 

More specifically, the Centre’s learning services are aligned with the ISO 29993:2017(E) Standard for learning services outside formal education, while non-training capacity development services such as communication and advocacy projects draw inspiration from the ISO 21502 :2020(E) Standard for project, programme and portfolio management.

 

The work of the Centre in the field of communication and advocacy is firmly anchored in its 2022-25 Strategic Plan and the 2022-23 Programme and Budget, which in turn draws inspiration from the ILO 2019 Capacity Development strategy and the UN common approach to capacity development.

 

The ITCILO P&B lists communication and advocacy projects as strategic means of action for capacity development under Output 1.3 Digitally enhanced support services for institutional intermediaries.

 

ISO 21502:2020 (E) specifies a set of integrated project management practices that “cover the practices to be used when undertaking a project from the pre-project activities leading up to the decision to initiate the project, through the planning and monitoring stages to the post-project activities”.

These integrated project management practices are performed in varying degrees by the sponsoring organization, the organization accountable for achieving the project objectives (in this case, the Centre represented by Technical Programmes in the Training Department), the project manager assigned by the decision-making unit and the work-package leaders reporting to the project manager.

 

project management PDCA cycle

The Project Management Cycle illustrated

 

 

For mapping ITCILO's current practices against the recommended ISO practices, two case studies of flagship communication and advocacy campaigns organized by the Centre have been chosen:

 

The Centre’s communication and advocacy services are mostly commissioned by institutional clients from the ILO constituency or by ILO development partners, such as UN agencies. The purpose, objectives and expected outcomes of these projects therefore typically align with the Centre’s broader mandate and arguably contribute to the achievement of its programmatic objectives.

For proof of concept, the Centre estimates upfront the project’s likely contribution to its own institutional technical and financial targets, expressed in qualitative and quantitative KPIs set as part of biannual programming cycles.

The financial and technical parts of each communication and advocacy project proposal are standardized, and financial standards are enforced by a Contract and Finance Committee. Compliance with technical standards is monitored by the Office of the Director of Training, and the final decision to go ahead is subject to approval by both the Director of Training and the Treasurer.

To initiate a communication and advocacy project, the Centre sets up a project team led by a project manager. The team will typically comprise technical experts, including a project manager from the Learning Innovation Programme and, depending on the nature of the assignment, technical experts from other Technical Programmes and even the project sponsor, often supported by consultants and external service providers.

Under the leadership of the manager, the team plans the project, defines the project organization, defines project governance and management, and identifies stakeholders.

To facilitate the planning activities in practice, the project team usually invites the key project stakeholders to a design-thinking workshop run with the help of the ITCILO Design Kit.

Detailed planning of the project, including task assignment and timeline development, takesvplace at this stage. The Centre uses various digital project management tools and methodologies to track task assignments and deadlines.

Depending on the complexity of the project, work is organized into work packages.

Depending on the complexity of the work package and the nature of the tasks, the work is assigned to a work package leader, either a technical expert from the Centre or an external resource person. In both cases, the work involved is by default defined and planned upfront in consultation with the project manager and monitored by way of a project implementation plan.

The Centre uses a range of digitally supported project implementation planning applications, adjusted to project complexity, budget and client expectations.

The purpose of oversight activities is to enable the client (sponsoring organization) to verify externally that the project objectives are being achieved within a dynamic and often changing context. Oversight focuses on key performance indicators linked to outcomes.

The Centre involves the sponsoring organization in key decisions by way of periodic online sprint meetings during which the project team reconvenes with the project stakeholders to review progress and, where applicable, modify the project implementation plan. The Centre also issues periodic updates on progress, validated against KPIs agreed upfront in the project implementation plan. More specifically and depending on the needs of the clients, the Centre issues progress reports or, alternatively or in addition, provides weekly data snapshots or sets up data dashboards to facilitate tracking of KPIs in real time.

Depending on the agreement with the client, the Centre might also carry out internal end-of project evaluations or facilitate ad hoc external reviews and audits of its performance.

 

The Learning Innovation Programme, in close consultation with the project stakeholders and - where applicable - after seeking clearance from The Office of the Director of Training, is directly responsible for ensuring that:

  • the project vision and objectives are being communicated with strategic assumptions, and criteria have been set for measuring the project’s success;
  • there is ongoing justification for the project and that the business case is being updated where applicable;
  • the solution, in terms of outputs, outcomes and expected benefits, is likely to meet the needs of the organization;
  • appropriate and competent resources are being used;
  • work is terminated when the organizational justification is no longer supported.

The purpose of controlling is to ensure internally that project implementation is following the critical path. Controlling involves the monitoring of a wide set of operational-level indicators.

The Centre uses the following result-chain model to monitor and evaluate the performance of its communication and advocacy projects:

Resultchain

the Centre makes a distinction between key performance indicators (KPIs) and operational-level/process indicators. KPIs track aspects of project performance that point towards effectiveness (doing the right things), while process indicators mainly track project efficiency (doing things the right way). Indicators can sometimes serve both purposes.

The choice of performance indicators at each stage of the results chain is project-specific. The same is true of performance thresholds, which are determined by project scope, ambition and budget. However, as observed in previous sections, selected KPIs are pre-set in all the Centre‘s communication and advocacy projects, cascaded from its institutional results-based management framework and intended to ensure that the project objectives align with the Centre’s mandate.

The project team verifies that all activities and work packages within the project scope have been completed/delivered and accepted by the sponsoring

organization. The closing activities usually involve a debriefing with the project stakeholders and the submission of a final technical report.

In addition, the project leader is to validate that all contracts for external resources and services have been satisfactorily completed, that all outstanding payments have been made and that all accounts receivable have been settled. Consequently, the closing activities might require the submission of a final financial report to the sponsoring organization.

Since 2022, as part of its annual reporting obligations at the post-project stage, the Centre has released aggregated statistics concerning the performance of its communication and advocacy projects, currently with the focus on KPIs related to inputs and outputs. The Centre carries out follow-up evaluations of project outcomes as and when contractually required by the sponsoring organization.

Depending on the nature of the agreement with the sponsoring organization, the evaluation findings are released to the public. The Centre also works with the stakeholders to share knowledge about the project outcomes and results. Subject to agreement with the sponsoring organization, these knowledge-sharing activities typically comprise of web articles and videos disseminated via digital media channels.

* At the Centre, the size of a communication and advocacy project can vary significantly across the portfolio, from small communication initiatives comprising few activities to multi-step advocacy campaigns linked to global ILO programmes. The PDCA cycle applies indiscriminately to each project, even though the level of effort required to meet each condition may differ considerably. Similarly, the level of monitoring and reporting might vary based on the size of the project.
4. Recommendations

 

The Centre should:

 

  • prepare a checklist for project managers including all the conditions set under ISO 21502; project managers would then have to document compliance in their project progress reports and justify exceptions
  • develop a project document template (aligned with ISO standards) for mandatory use at the project initiation stage
  • draw up a list of process indicators and KPI for use in communication and advocacy projects, and highlight which of these metrics are mandatory
  • classify default digital work packages for use in its communication and advocacy projects
  • aim for multi-level and cross-communication-channel engagement and allow for rapid iteration of channel mix in response to feedback
  • introduce a formal mechanism to direct communication and advocacy projects by adding a governance layer to flagship projects
  • commission annual external sample evaluations of its communication and advocacy projects
  • put even stronger emphasis on story-telling as a means of capturing qualitative impact rooted in evidence
  • develop a theory of change that explains how its channel-specific capacity development activities compound to promote global social justice at the impact stage