Quick Search
    

Training

Featured Text

 

ROLE 2: Assess Performance Information:

Practice 2b. Assure performance reports: Audit, attest to, assure, or certify external performance reports.

British Columbia Auditor General [Profile]

The Budget Transparency and Accountability Act (BTAA), passed by the Legislative Assembly of British Columbia (BC) in 2000, highlights legislators’ desire for more meaningful and rigorous public scrutiny of government performance.  The BTTA requires ministries, government organizations, and the government as a whole to provide three-year service plans and report annually on their performance against their plans.   To help ensure the reporting of meaningful performance information, the Office of the Auditor General (OAG) assessed the quality of performance reporting contained in the annual reports of a cross-section of the government’s ministries and Crown corporations in each of the five years subsequent to the BTAA coming into force.  Results of these assessments were published in the “Building Better Reports” Series (BBR).  Initially, assessment results were published on an aggregated basis, although results were made available to individual organizations.  Examples of best practice were highlighted so that organizations could learn from the practices of others, though over time this emphasis decreased in favor of the publication of individual assessments.  Reports were not explicitly ranked in order of assessment scores, although interested organizations could easily see how their assessment compared to others.  Each year, between 10 and 20 staff from across the office participated in assessment teams for periods ranging from 2 to 8 weeks, depending on the number of reports assessed by each team, with one assessment coordinator assigned for the duration of the project.  In total, 3,000 staff-hours per year were devoted to this practice, including meeting with report preparers from each organization to assess results of OAG reviews with them.

Over the 5 years covered by the Building Better Reports series, performance quality was assessed using a “learning model” approach (see “tools” below) that showed progress over four stages of development for incorporating good reporting principles: “Start-up,” “In Process,” “Fundamentals in Place,” and “Fully Incorporated.”  The OAG developed eight assessment scales, in matrix form, describing detailed characteristics for each of the four developmental phases for the following eight principles of a good performance report:

  • Explain the public purpose served;
  • Link goals and results;
  • Focus on the few, critical aspects of performance;
  • Relate results to risk and capacity;
  • Link resources, strategies and results;
  • Provide comparative information;
  • Present credible information, fairly interpreted;
  • Disclose the basis for key reporting judgments.

When the OAG assessed an organization’s performance report, it summarized the results in an assessment matrix showing the development stage for each performance reporting principle assessed. In 2003, after deliberation by a high-level steering committee, these eight principles were endorsed by the BC Legislative Assembly’s Select Standing Committee on Public Accounts (PAC) “as a basis for service plan and annual service plan report guidance and assessment.” The  PAC also encouraged “the Auditor General and Government to revisit the principles at a future date to ensure that BC’s reporting principles continue to reflect good reporting practices in Canada and internationally.”  Furthermore, a resolution was passed by the PAC to encourage “Ministries and Crown Agencies to work towards the goal of incorporating into their contracts with non-ministerial service delivery agencies the eight principles.”  The OAG made the assessment scale matrices and methodology, as well as information on the PAC’s endorsement available to the public through use of both published and web-based reports.  One such report, Reporting Principles and An Assurance Program for BC (March, 2003)also included self-assessment criteria that government organizations can use for assessing their own public reporting (see “tools” below).

From 2002–2006, the Auditor General used the same reporting principles as used in multi-organizational reviews to issue separate assurance reports on the annual performance reports of the British Columbia Public Guardian and Trustee (PGT).  These include assessment of the relevance and reliability of the performance measures, and each project takes about 650 staff hours to complete.  Similar assurance reports have been provided for the performance reports of the Workers’ Compensation Board of BC (WorkSafeBC) since 2004 and the British Columbia Assessment Authority in 2006.  The OAG has received strong positive feedback in relation to its assurance engagements and WorkSafeBC, BCAA, and PGT. The reports of these three organizations are among the strongest in British Columbia’s public sector.

After five years, the OAG felt that the BBR series had begun to have less of an impact than they had initially, and that the quality of reporting had reached a plateau, especially in the case of the ministries.  As a result, the OAG now devotes less time to the cross-government assessment of report quality, though it may become a priority again in the future.  For 2006, the OAG is preparing a retrospective report examining the trends in performance reporting over the previous six years, and identifying the opportunities for improvement that remain.

Recognizing that learning phases must inevitably give way to a performance phase, in 2006 the OAG discontinued the “learning model” approach described above in favor of a “threshold approach” (See “tools” below). The quality of reports continues to be assessed on a principle-by-principle basis.  However, rather than placing the results of these assessments on a rating scale, assessors are asked to conclude whether or not the principle has been fundamentally met.  This standard is consistent with the expectations for a “fundamentals in place” assessment in previous learning model, and recognizes that the spirit and intent of each reporting principle can be achieved without 100% compliance with all of the detailed self-assessment criteria.

More information on the Building Better Report series and multi-organizational reviews [Example in 2004 Guide-PDF]


Tools:

Reports: