How to deal with corruption metrics like the CPI

CORRUPTION in the Philippines is perceived to have further worsened as the country sank to its lowest ranking to date in the Transparency International‘s Corruption Perceptions Index (CPI) for 2008.

From 131 (out of 179 countries) last year, the Philippines is now at 141 (out of 180), down 10 notches with a score of 2.3, the worst it has attained since the composite index, which draws from several expert and business surveys, was introduced in 1995. The country’s CPI score has a confidence range between 2.1 and 2.5.

The CPI score relates to perceptions of the degree of corruption as seen by business people and country analysts, and ranges between 10 (indicating low levels of perceived corruption) and zero (indicating high levels of perceived corruption). The confidence range, meanwhile, provides a range of possible values of the CPI score and reflects how a country’s score may vary.

Under Gloria Macapagal-Arroyo, the country has been steadily slipping in the TI Index, with the country’s CPI score unable to improve beyond 2.6. (see table)

Source: Transparency International

Perennially outscored by Southeast Asian neighbors Singapore (9.2), Malaysia (5.1), Thailand (3.5), and Vietnam (2.7), the Philippines was even surpassed by Indonesia (2.6) this year. The country’s score was better only compared to Timor-Leste, Bangladesh, Laos, Papua New Guinea, Cambodia, Afghanistan, and Myanmar in the whole of Asia.

Across the Asia-Pacific region, TI noted that corruption and lack of transparency, particularly in political financing, “clearly remain serious challenges.” As indication of the serious corruption problem in the public sector, only ten of 32 countries in the region scored above 5, while 22 scored below 5.

As it has been wont to do, the Arroyo government dismissed the latest TI survey findings as mere “perception” and “not Gospel truth.”

It begs asking though, with the Philippines’s deteriorating CPI score over the years, if corruption measurements like the more established and widely used index of TI have any value at all to the government, particularly in informing its efforts to address a problem that is increasingly seen as a major stumbling block to a country’s development.

The obvious reply, especially among critics of the Arroyo government, would be a big “No,” pointing as they do to its “lack of political will” to go after corrupt officials, moreso if they happen to be its political allies.

While the likes of Jocelyn ‘Joc-joc’ Bolante, Virgilio Garcillano, and Benjamin Abalos Sr., to name only a notorious few, provide ample basis for such conclusions, a new publication released a week ahead of the TI survey does acknowledge that there are also challenges that anti-corruption and governance practitioners face with existing corruption indicators like the CPI.

In “A Users’ Guide to Measuring Corruption,” which was jointly produced by The Global Programme on Democratic Governance Assessments and Measurements housed at the UNDP Oslo Governance Center and Global Integrity, interviews with researchers, donor officials and government policymakers identified certain gaps of corruption indicators. These include:

• the lack of corruption metrics that are useful in day-to-day policy and programmatic work;

• the need for more disaggregated data that move beyond single-country rankings to more discrete measures within sectors, institutions and population groups;

• the need to move beyond perceptions-based data as the basis for corruption measurement.

Malacañang may have a point when it questions the country’s annual CPI score since, like most existing corruption metrics, the TI index relies on subjective indicators centered on citizens’ or experts’ perceptions and opinions about the level of corruption in a respective country.

As Presidential Anti-Graft Commission (PAGC) chair Constancia de Guzman puts it, the TI report is not based on a “factual assessment of the actual situation. As such, it does not reflect the realities on the ground.”

A common complaint among government officials and advocates working to improve governance, according to the guide, is exactly that: the “frustrating disconnect between early but important steps toward fighting
corruption, and the sometimes fickle moods of popular opinion.”

Download the Users’ Guide to Measuring Corruption.

It is argued that even if perception-based measures are perfectly effective in tracking public opinion, and even if public opinion is perfectly responsive to changes in levels of corruption, the results of these measures are “still rather limited in their application, since assigning a single number score to an entire country yields little insight into potential solutions.”

As the more than 30 anti-corruption and governance practitioners interviewed for the guide point out, there are potential drawbacks to using subjective indicators. Four such disadvantages are mentioned:

• First, subjective indicators are based on perceptions and may not be reliable when assessing long-term trends and changes. Improvements in the quality of a country’s public integrity system and anti-corruption performance are difficult to capture.

• Second, indicators that hinge on perceptions often lack credibility because of the dearth of de jure facts and the gap with de facto realities on the ground as experienced by the public.

• Third, most subjective indicators are skewed toward the perceptions of the elite business community and may not always align with the views of non-business people and ordinary citizens.

• Fourth, these indices tend to gauge perceptions of governance outcomes or corruption, rather than their causes.

And as with other indices, the CPI, they say, is also not clear as to the types of corruption perceived and how they are measured.

But that is not to say that subjective indicators are entirely wrong. As the guide advises, objective data is not necessarily better than subjective ones. In the case of the CPI for this year, TI drew from surveys of business people and assessments by country analysts which are acknowledged as credible. All in all, there were 13 sources originating from 11 independent institutions, including the Asian Development Bank, World Bank, Economist Intelligence Unit, Freedom House, Political and Economic Risk Consultancy, and the World Economic Forum.

What is important, the guide says, is for practitioners to bear in mind the extent to which indicators rely on subjective versus objective data as a methodological distinction.

The Users’ Guide further advises:

Also bear in mind that there’s no right answer to the ‘objective vs. subjective’ or ‘composite vs. original’ arguments. Varying assessments and corruption measures can be useful, depending on the context. If a user is interested in a simple snapshot of country performance relative to its neighbors, then a high-level composite indicator is likely sufficient. Similarly, if a user is attempting to assess citizens’ views on the climate of corruption, then a purely subjective public opinion survey is entirely appropriate. On the other hand, if a user is sitting in a national government office and charged with improving country performance on anti-corruption, those sources will likely need to be complemented with more actionable and objective indicators that offer specific entry points for reform, which mirror or address the sentiment reflected in public opinion surveys or international composite indicators.

With the proliferation of many indices, toolkits and corruption assessments in the last decade, anti-corruption and governance practitioners will find the Users’ Guide’s evaluation of existing corruption indicators useful. The discussion — which looks into the areas of the scale and scope of indicators, what is actually being measured, methodology employed, and the role that internal and/or external stakeholders play in generating the assessments — will help them better understand the methods and objectives of each measurement tool in addressing their specific governance and corruption challenges.

Not only does the User’s Guide identify the most pressing issues faced by users of corruption measurement tools and how these can be further improved, it also offers government, civil society and development practitioners with “good practices” and case studies in measuring corruption.

Good practices, says the guide, include:

• knowing what you want to measure, and finding the appropriate measurement tool;

• breaking down ambitious goals of “measuring corruption” into more discrete and clearly defined policy outcomes;

• focusing on “actionable” data that can inform policy choices;

• looking for data that capture the voices and experiences of the poor and minority groups;

• combining quantitative data with qualitative political-economy analysis;

• engaging national actors and use local sources of information whenever possible; and

• being transparent in constructing a methodology for measuring corruption.

This publication is the first in a series of thematic users’ guide to be produced by the UNDP Global Programme. The series will offer guidance to assessing and measuring specific thematic area of governance, such as corruption, local governance and public administration reform, aimed at strengthening national governance assessments at the country level.