Managing complicated public projects

Managing complicated public projects

Nothing unfamiliar, but the way Clay Shirky describes it is excellent. In the context of the problems faced during the launch of the HealthCare.gov insurance portal in the US, he describes the challenges faced by technology projects,

On a major new tech project, you can’t really understand the challenges involved until you start trying to build it. Rigid adherence to detailed advance planning amounts to a commitment by everyone involved not to learn anything useful or surprising while doing the actual work. Worse, the illusion that an advance plan can proceed according to schedule can make it harder to catch and fixed errors as early as possible, so as to limit the damage they cause. The need to prevent errors from compounding before they are fixed puts a premium on breaking a project down into small, testable chunks, with progress and plans continuously reviewed and updated. Such a working method, often described as “agile development,” is now standard in large swaths of the commercial tech industry.

The larger a tech project is and the more users it will have, the likelier it is that unexpected bugs will surface. And the longer term a technological prediction is, the likelier that it is wrong. A technology plan that tells you what will be happening next week is plausible. One that tells you what will happen next year is far less so. One that tells you what will happen in five years is largely fiction. So thinking of a tech project as something that can be implemented according to a single, fixed plan, with a product that can be delivered in a package at some fixed date long down the road, can be a recipe for disaster.

Each step of a tech project’s implementation thus serves three functions. The obvious function is bringing the project further toward completion. But two other functions are also essential: any step in the implementation tests the assumptions that went into the design, and it produces new information that can and should be used to inform planning for the rest of the project. The people who want to be able to procure technology the way they would procure pencils often ignore both of those informative functions... One might think that detailed advance planning would be extremely helpful in this regard, but in fact, what overly meticulous planning actually does is trade away flexibility long before it is necessary, making it harder, rather than easier, to handle unforeseen problems as they inevitably arise...

Each step of a tech project’s implementation thus serves three functions. The obvious function is bringing the project further toward completion. But two other functions are also essential: any step in the implementation tests the assumptions that went into the design, and it produces new information that can and should be used to inform planning for the rest of the project. The people who want to be able to procure technology the way they would procure pencils often ignore both of those informative functions.

About excessively ambitious and optimistic project planning and implementation schedules, 

Assuming basic technical competence, the essential management challenge for all large technology projects is the same: how best to balance features, quality, and deadline. When a project cannot meet all three goals simultaneously... something has to give, and management’s job is to decide what. In such cases, if you want certain features at a certain level of quality, you have to move the deadline. If you want overall quality by a certain deadline, you have to simplify, delay, or drop features. And if both the feature list and the deadline are fixed, quality will suffer, and you have to launch and fix after the fact. 

I think much the same applies to all complicated projects, ones where there are too many "unknown unknowns". They span the full spectrum to running good schools and hospitals to rolling out large transportation projects like a new airport of a mass transit service. The challenge is compounded by the inherent nature of large bureaucracies - with its limited tolerance for failures, impersonal reporting systems, limited operational flexibility, aversion to taking judgment calls, incentive incompatibility of managers, and so on. Navigating this challenge is not easy and explains why many ambitious projects bite the dust. 

Getting legislators to legislate

Getting legislators to legislate

An oft-repeated complaint among bureaucrats is that legislators interfere too much in operational issues like transfers of field functionaries and procurement of goods and services. It is common knowledge that the MP or MLA have strong vested interest in the postings of officials deputed to work in their constituency and the contractors awarded local works or supply orders. Legislators put pressure on officials to violate rules to accommodate these vested interests. It forms a major source of patronage and a channel to keep their loyal followers satisfied. But this interference in routine administrative activities contributes to the politicization of grass-roots administration as well as the erosion of state capability.

 

Why do law-makers end up becoming rule-breakers? The simple answer is that of patronage, corruption, and self-aggrandizement. A more sophisticated answer is that it is a reflection of responsibilities and incentives within the system.

The unfortunate reality is that legislators, or law-makers, spend very little time on their primary responsibility of making laws. Though Parliament and Assemblies discuss and legislate laws, much of that work is notional. The major part of the work of preparing legislations is done by the concerned Ministry, at the instance of the ruling coalition. The individual legislators have a very limited role in this process. Apart from their occasional intervention, if any, in the discussions that take place in the legislature when the bill is tabled, the legislator could be nominated to a Parliamentary or Assembly Standing Committee that are sometimes constituted to scrutinize the Bill. Their other statutory responsibility is participation in the activities of the regular legislative committees. A legislator is typically a member of one such committee and his time commitment for this is minimal.

Here, the nature of India’s parliamentary democracy differs from that in countries like the US. In the US, legislations are brought forth as private member bills, where a handful of legislators, often cutting across party lines, come together and formulate bills which are then brought before the legislature. While the provision of private members bills exists in India too, it is rarely used as parties front legislations. 

Accordingly, legislators in US have large office establishment to facilitate the process of preparing legislations. A typical Congressman has an official establishment with 40-60 staff members, the MP in India has to rely on just 1-2 official assistants. The situation is even more perilous with MLAs. In the absence of any support staff, it is extremely difficult for even well-meaning and committed legislators to contribute meaningfully to making laws or even plan for the long-term development of their constituencies.

Deprived off any substantial role in their primary role of law making, legislators expend their energies on activities within their constituency. This generates two negative externalities. One is the proclivity to interfere in administrative matters at the district and constituency levels. The other involves competition and conflict with their counterparts in the district, block, and village panchayats. The latter becomes especially pronounced in cases where they belong to different political parties. The turf battles in turn ends up exacerbating the politicization of administration and weakening of public systems.

So how do we get Indian legislators to become meaningful participants in the process of development?   

India's "bloated" bureaucracy?

India's "bloated" bureaucracy?

Critics reflexively blame India's "bloated bureaucracy" for governance failures. So how "bloated" is it?

As on 2011, India had 18.5 million governments employees - 3.4 million with central government, 7.2 million with state governments, 5.8 million with PSUs, and 2.1 million with local governments. As a share of total employment (around 400 million) it was just 4.6%. However, as a share of organized sector employment, it stood at 62.2% in 2010. In contrast, in the US, governments now employed 15.9% of all Americans with jobs. The relevant statistical comparisons are 4.6% and 15.9%, which is itself a deceptively favorable comparison and with wide variations across states and departments. Consider this excellent analysis reported in Hindu, 

Data compiled from multiple sources, including a 2008 official survey, Right to Information applications, media reports and the 2011 census show, India has 1,622.8 government servants for every 100,000 residents. In stark contrast, the U.S. has 7,681. The Central government, with 3.1 million employees, thus has 257 serving every 100,000 population, against the U.S. federal government's 840. This figure dips further if the 1,394,418 people working for the Railways, accounting for 44.81 per cent of the entire Central government workforce, are removed. Then, there are only about 125 central employees serving every 100,000 people. Information technology and communications services account for another 7.25 per cent of the Central government's staff...

For the most part though, India's relatively backward States have low numbers of public servants... Bihar has just 457.60 per 100,000, Madhya Pradesh 826.47, Uttar Pradesh has 801.67, Orissa 1,191.97 and Chhattisgarh 1,174.62. This is not to suggest there is a causal link between poverty and low levels of public servants: Gujarat has just 826.47 per 100,000 and Punjab 1,263.34. The data could explain, though, why even well-off States like these have found it tough to ensure universal primary education and eradicating poverty.

I have always considered this alarming deficiency as an important contributor to our state capability deficit. An illustration of this deficit is that New York with 3500 eateries has 180 food inspectors whereas Hyderabad with atleast a few times more eateries has just 4! Much the same is common across cutting edge activities - school inspectors, outreach nurses, agriculture extension officers, town planning inspectors, engineering supervisors, and so on. It is impossible to systemically deliver effective outcomes when the jurisdiction of the official is massively stretched functionally, geographically, and population-wise. Even a cursory matching of time and task to responsibilities would reveal that we would need to have a few times more functionaries.

And, as the report highlights, these deficiencies span across all levels. It is as much incorrect to say that Indian bureaucracy is top heavy as it is to say it bottom heavy. Or overstaffed at the center and understaffed at the states. It is understaffed at all levels. 

None of this should be taken to presume that once we have them in place, state capability will improve. Far from it. Personnel deployments have to be complemented with other administrative reforms that increase transparency and local accountability, improve supervision and monitoring capacity, build capacity, and prevent the politicization of bureaucratic functioning. None of these are easy. But to mindlessly blame "bloated" bureaucracy for our governance failures will not get us anywhere.