Last week, Transparency International Pakistan released the National Corruption Peception Survey 2011. You have heard about the report and its results, no doubt, from one of the many news reports about its findings. Of particular interest has been its findings about corruption in the military as this is the first year military was included in the survey. What you probably haven’t heard much about, however, is that the report is garbage.
Before parroting the findings of any ‘report’, people should take a close lookat the study’s methodology. According to the methodology explanation for TIP’s latest report, Gallup Pakistan (not related to Gallup Inc. headquartered in Washington D.C. USA) surveyed 2,500 Pakistani men and women adults. The error margin for a sample of this size is ±3.5%.
According to the results, only 3% of those surveyed said they had any contact with the military. Of that subset, 11% said they “felt compelled to pay a bribe”. First of all, 3% of respondents is already within the margin of error. But let’s take it a step further: 11% of 3% of people answered in a way that suggests the military is corrupt That’s approximately 8 people only. Which tell us, essentially, nothing. It should also be noted that only 4% of those surveyed said they had any contact with either Customs or Tendering & Contracting.
Let’s also consider the questions that were asked by TIP. Actually, there were only two: First, “In the recent 12 months, did you or your family get a chance to contact any of the following institutions or not?” and second, “Did you feel compelled to pay a bribe?”
Nobody asked why they felt compelled to pay a bribe. Nobody asked if a bribe was actually demanded. Nobody asked if this hypothetical bribe were accepted. And nobody seems to have considered that those people who had contact with institutions like the military might not want to tell a complete stranger who called them the phone that they ‘felt compelled to pay a bribe’ and thought an institution like military was corrupt. With regard to the Education department, does ‘tuition’ count as a bribe? How was this explained to the survey participants?
It is also important to look at the different institutions that TIP asked about. Obviously more people are going to have more contact with Police, Electricity Supply and Taxation than with Military or Tendering & Contracting. Comparatively, these numbers are meaningless. Even if the answers did fall outside TIP’s own margin of error, which they don’t, it appears from the methodology that they asked 2,500 people if they had any contact with each of the 10 institutions – they didn’t find 2,500 people who had contact with each institution. Just because someone has contact with the military and ‘feels compelled to pay a bribe’ but didn’t ever have contact with the police, it doesn’t mean that they think the military is more corrupt than the police. That’s a false inference. And what about Education? Do people consider ‘tuition’ to be a bribe? The ambiguity of the only two questions asked make the answers meaningless.
Furthermore, you can’t rank these institutions because the samples are completely different and the answers are ambiguous. You would be comparing apples to oranges. But that doesn’t stop TIP, who terms Police as most corrupt and Education and Military as least corrupt. Maybe this sounds plausible, except that none of that is supported by the data in this report.
Also worth noting, the institution with the greatest percentage of public contact was Health Department, and even that is below 40%! According to TIP’s study, up to 61% of the population may have had no contact with any government agency at all since the past 1 year!
As far as I can tell, the only thing TIP’s report is good for is as a talking point to criticise government and security agencies. Of course, people will point out that there is a lot to criticise. But this report offers nothing in the way of useful information that could possibly help to eliminate corruption in society. Rather than generating junk studies like this, TIP should produce legitimate research so that we can learn about the problems plaguing the nation and figure out how to actually do something about them.
Lastly, I want to note one thing that always drives me crazy about these Transparency International reports. They are not studies of corruption. They are studies of perceptions of corruption. I don’t know how much co-Sponsors Swiss Agency for Development and Cooperation SDC and USAID paid TIP to tell them that people think corruption is a problem, or that people pay bribes to police, but I could have saved them a lot of money. Most frustrating, though, is that these reports are self-reinforcing. People read reports about the annual TI report saying corruption is a problem. So they believe corruption is a problem. Then next year TI calls and asks them if corruption is a problem. “Yes, of course, I read your study!” With “research” like this, is it any wonder we’re not making any progress?