By Stuart Grifel, CIA, CGAP, Principal, Auditor Roles Project
In preparing the city’s first annual Service Efforts and Accomplishments report (released in January 2009), the San Jose City Auditor’s Office surfaced a number of systemic performance management issues. To address these issues, the City Auditor’s Office reviewed performance management more thoroughly, and prepared the report “Performance Management and Reporting in San José: A Proposal for Improvement (PDF),” with a cover letter jointly signed by the City Auditor and City Manager. The report recommends “next steps” to help transform the city government into a data-driven, high performing organization. The next steps are meant to reduce staff time compiling data while ensuring city staff and policy makers use the best information available for decision making and increasing accountability and transparency in the city’s public reporting. As some of the next steps are implemented, the City Auditor is likely to become more involved in using several exemplary practices featured here at the Auditor Roles website.
Background: A Model System Not Consistently Maintained
When San Jose developed a performance management system in the 1990s, the Governmental Accounting Standards Board and International City/County Management Association considered it innovative and published case studies on the San Jose system as a model for other local governments. But over a ten-year period, changes in management staffing and priorities led to inconsistent system maintenance and use across departments and programs. That’s what audit staff found, as did a 2008 consultant study which said that the system in place at the time accomplished performance measurement but not performance management. The consultant study was part of the City Manager’s “Beyond Budget Cuts” efforts to make city government more efficient and effective. So, the City Manager was interested in supporting a joint proposal with the City Auditor to make systemic performance management improvements.
Highlights of Findings of the Auditor’s Performance Management Review
Key performance management findings from the City Auditor’s report included:
- The City currently reports performance measures, but these are not consistently used as a tool to improve performance
- Performance management is occurring in isolated programs or departments, but not in a broad, city-wide manner
- Executive management is not routinely involved in discussions about performance measures.
- Many measures did not meet city criteria for performance measures (below).
- The titles of a number of core city services do not consistently describe what services are provided, which can make it difficult for users of performance information to find information that most interests them.
- A number of city services have multiple levels of missions, goals and objectives, making it difficult to know how services and performance measures are aligned with the city’s priority goals. Thus, it would be difficult for an outsider to know what the most critical priorities for the city are, and who is accountable for achieving stated goals or outcomes.
- Citywide expenses were not allocated to individual departments or core services and not all program expenditures from certain operating funds were allocated to individual departments. The effect is that it is difficult to ascertain the full scope and cost of department services and who is responsible for certain expenditures.
City of San Jose Criteria for Performance Measures
The City of San Jose’s criteria of selecting performance measures emphasizes that the measures should be “meaningful, useful, and sustainable” defined as:
- Meaningful – Understandable to internal and external stakeholders; based on goals and objectives related to an organization’s mission and purpose; focused on controllable facets of performance.
- Useful – based on reliable data; accurately assesses performance; comparable to other periods or targets; reported to the appropriate level and to the appropriate audience.
- Sustainable – ease of collection; value of the data does not exceed the effort to collect the data.
Recommended Next Steps, by Section of the Report
Section I: “Moving San José from Performance Measurement to Performance Management” includes these next steps:
- Develop a performance management system with the following characteristics:
Ongoing, periodic meetings that include management to discuss performance measures and use data to analyze departments’ past performance, establish the next performance objectives, and examine overall performance strategies.
- Set minimum expectations for performance management at the department level.
Section II: “Ensuring that Performance Measures are Meaningful, Useful, and Sustainable” includes these next steps:
- The City Auditor’s Office will assist the Budget Office in training teams to design performance measures that are meaningful, useful, and sustainable. Helping departments design or choose better measures is an exemplary practice performed by other innovative government auditors.
- Methodology sheets explain the performance measure’s meaning, its data source and how it is calculated.
- Have the City Manager’s Office and the City Auditor’s Office coordinate efforts to ensure that departments only submit year-end data at one time and in one place to ensure consistency and reduce staff time spent compiling data.
- The City Auditor’s Office will also be incorporating audit steps in their regular performance audits to test whether the performance measures for audited programs are meaningful, useful and sustainable. Validating and testing performance measures is also an exemplary practice performed by other government auditors.
- Implement automated solutions to collect performance measures.
Section III: “Clarifying Core Services and How They Align with Organizational Mission, Goals, and Objectives” includes these next steps:
- Clarify core service names
- Clarify the link between missions, goals, and objectives. Budget Office meetings with executive management to discuss performance should include clarifying the links between the city’s core services, CSA (City Service Areas) outcomes and strategic goals, CSA priorities/key services, department missions, and CSA missions.
Section IV: “Gauging Whether Performance is Cost Effective” discusses problems with not allocating expenses to department or core services, includes these next steps:
- The Budget Office should explore ways to include and/or cross-reference more information about revenues/disbursements, citywide expenses, program expenditures from enterprise and special revenue funds in individual department budget sections and CSA sections in the operation budget.
- Allocate strategic support. To the extent possible strategic support costs should be allocated to individual core services so that a true full cost can be determined.
- Use efficiency measures. The City should incorporate full or net cost information into its data-driven decision making.
Section V: “Moving Forward”
Section V of the report, “Moving Forward,” discusses the role that performance measurement can play in helping to address the City’s structural deficit. The 2008 “General Fund Structural Elimination” plan included an analytical framework for service reductions and eliminations. Key elements of that analysis included performance impacts and net cost-items discussed in the audit report.
Section V also addressed the evolving role of auditors and how the auditor practice of assisting management in implementing or improving the City’s performance management systems will be carried out at the citywide level by the City Auditor’s Office. This practice is another way that the City Auditor will be following in the footsteps of other innovative government auditors who have been assisting management in improving performance management. In San Jose, this could include providing background information and advice on the development of implementation guidelines, training staff, or advising staff in the selection of performance measures.
Follow-up Auditor Projects
Ongoing assistance to the City Manager’s Office in the development of a performance management system, assisting a high performing work team by verifying performance measures and costs, and the compilation of the Service Efforts and Accomplishments Report was part of the Audit Office’s 2009-2010 Audit Work plan.
For example, in January 2010, the City Auditor issued San Jose’s second annual “Service Efforts and Accomplishments Report” (PDF 2 MB). And in March 2010 the City Auditor published an audit report titled “Review/Validation of Sewer Line Cleaning Performance and Cost Data” (PDF). The purpose of the audit was to review and validate baseline information related to the performance and costs associated with the Sanitary Sewer Line Cleaning Program. This program was identified by the City’s “Beyond Budget Cuts” to empower employees through team training and performance model development as a way of improving performance and reducing costs over a multi-year period.