What the quality of bureaucracy can tell us about government effectiveness

Guo Xu, Associate Professor at the Haas School of Business, joined the O-Lab Faculty Network in May 2024. Xu joined Haas in 2017, where he researches the organization and development of personnel in both public and private-sector organizations, working at the intersection of political economy, economic development, and economic history. Much of his recent research has examined aspects of bureaucracies, exploring topics including the interplay of political ideology and bureaucrat performance, how large shocks influence norms in the civil service, and the role of bureaucracies in economic development.

We sat down with Guo and discussed key takeaways from his work on bureaucracies and personnel, how he has leveraged data to inform multiple investigations into economic history, and where he’d like to take his research next.


How did you become interested in economics research? Were there any particular policy problems or phenomena that you witnessed that initially caught your attention?

I'm a trained development economist, so the question I always had was: why are some places rich, and others poor? When you travel around, you notice these huge income differences – what's driving that? During my undergraduate studies, it was all about institutions – there were all these papers on colonial legacy, and the impact stemming from differences in institutions that were set up. I found that really fascinating. At the same time, I wanted to dig deeper, which led me down the rabbit hole of looking at public organizations and their personnel.

Your primary research fields of interest are political economy, economic history, and development economics. Can you describe your specific research interests within these fields, personnel being one of them?

My work is really at the intersection of these fields. The questions I have in mind tend to be development questions. There's no denying that you need some basic set of public goods for a country or society to function. So a natural question is, “How can you improve public service delivery to stimulate economic development and growth?”

The intersection with political economy comes into play when thinking about policy. Sometimes you know what the right policy choices are, and yet they are not implemented for political reasons. And even if policies get implemented, there remains a huge amount of variation in the quality of implementation. You might see the same policy being rolled out within the very same country - and yet there will be differences depending on how effective the local administration is.

The historical part comes in because these studies often rely on natural experiments such as big reforms or large-scale changes in governance that happened in the past. What's very appealing with the study of history is that you can rewind time and study big events after the dust has settled.

Much of your recent research centers on studying bureaucracy, with evidence from the US Federal Government. Can you describe your interest in studying workplaces of the public sector, as compared to the private sector?

A lot of the toolkits that I use are also used in the study of private sector organizations. Instead of studying senior bureaucrats, you study CEOs and managers. There are many similarities, but what I find particularly appealing about the study of public organizations is simply that they're so big and important for the lives of many. In the U.S., for example, the federal government has been the biggest civilian employer for a long time. The public sector is a huge part of the economy.

Could you go over some of the key takeaways or main findings that you've found from looking at the public sector?

There's often this notion that management in the public sector is different from the private sector. But most personnel practices in the private sector also apply in the public sector. For example, some might say that monetary incentives in the public sector don't work, or that paying people more is not necessarily a good thing – you might crowd out highly motivated people. But most of the research seems to suggest that those differences are not really there once you look closely. Many personnel policies that we've seen in the private sector seem to work, to some extent, in the public sector as well.

To give you an example, there’s typically a notion that bureaucracies are very rigid, and can't really motivate people, because you don't have the same type of systems - performance pay or up-or-out promotions - in place as in the private sector. In a study where we looked at the Indian federal bureaucracy, which is particularly rigid, we see that incentives matter quite a bit, and that there are always margins for incentives to kick in. In that particular setting, it turned out it was not about providing monetary incentives, but promotion incentives that allow individuals to reach the most prestigious positions - the “glittering prizes” - before the mandatory retirement age. This implicit retirement incentive governs how people exert effort throughout their career. There are many indirect ways of providing incentives – for example, by transferring employees to more desirable locations or more prestigious postings. Even rotations – you don't have to promote people, you can rotate them across different jobs – can get you quite far.

I expect that one key advantage of doing research on the US government is the availability of data, but in the case of your paper on federal employment segregation, you undertook an extensive digitization project. Can you describe what this process looks like? It seems like you've been able to use that data to study a couple of different questions.

One major part of doing economic history is trying to unearth old data. At that time, we wanted to study the economic costs of the introduction of racial segregation within the federal government under Woodrow Wilson. We were lucky to find these personnel records that go far back in time. 

Getting historical data into a workable format is a lot of work – back then, we relied on optical character recognition, and then processed the data in a semi-automated way. We did a lot of manual work too – we had teams of people check the data line-by-line. That was only a couple of years ago - I’m curious to see how AI will help speed up these digitization efforts.

Back then, however, it was a pretty big investment in terms of data collection. It took us more than a year to digitize and clean it all. Luckily, we were also able to use the data to analyze other questions. One question that we got from the study of federal employment segregation was: “Why are there so many black civil servants in the first place?” It seemed very unusual because of the ongoing discrimination at that time. That led us to study the introduction of the Pendleton Act a couple of decades earlier.

Before Pendleton, the President could appoint public servants at will - everyone served “at the pleasure of the President.” The Pendleton Act put in controls by introducing rule-based selection, and employees couldn't be fired at will anymore. We studied that rollout, and looked at whether it actually changed the composition of workers and their performance. The idea was that the introduction of rule-based selection would remove potential biases, increasing the entry of individuals from underrepresented groups. One surprising thing we found was that the Act didn't do much to change the demographic makeup of the civil service. So what we thought might be the primary effect of the Pendleton Act turned out to be much less important.

But it turned out that we still saw large performance gains, which we traced to the fact that the Act was reducing disruption in the organization. In the old system of appointments, with every new presidential election, the President would replace existing civil servants with their own political supporters, creating huge layoff cycles every 4 years. With service protections in place, these political cycles began to diminish and ultimately disappear. That, in turn, made the organization more stable, creating a civil service that more closely resembles the stable careers we associate with public sector work today – you can transmit knowledge more, and there's more on-the-job learning. We found that this increased the performance of the public sector, as measured through reductions in postal delivery errors and increased cost efficiency.

We focused on post offices for the study for several reasons: first, the Pendleton Act was gradually rolled out across post offices, allowing us to compare post offices with newly introduced civil service protections to those without. Second, rich administrative data on post offices was plentiful, allowing us to apply modern statistical methods to estimate the effects of the reform. And of course, the post office was a crucial infrastructure through which communication flowed, connecting far-flung areas of the country.

Initially, we expected the reform to have a large impact on getting individuals from underrepresented backgrounds into the government - but it turned out that the question of whether exams would increase the amount of Black employees in the civil service was not that clear, even in the historical record. It was a case where doing empirical analysis yielded some interesting surprises.

How do you see retrospective research as informing future policy?

It's perhaps a little cliché to say, “Oh, you can always learn from the past” - but that’s true. We can’t run experiments all the time, and many policies and events take a long time to fully unfold, making them difficult to study. In that sense, history offers useful episodes and lessons we can look at from a more distant perspective. 

That said, I think economists tend to be wary of extrapolating, unless we fully understand the underlying mechanisms through which certain effects unfold. All empirical research is retrospective and in a sense historical. Historical work is not about the age of the data, but taking a deep dive - a careful study of the context and institutional setting. This also helps to get a sense of how comparable different events in time are and the extent one can extrapolate.

Personally, I get a lot out of studying history, because it's a great way to learn – when I moved to the US, I didn't know much about US administrations. The good thing about this job is that you can choose what you study, and that's very enjoyable.

What kinds of new questions have your research projects raised that you see as important areas for future projects?

The nice thing about studying public organizations is that you can study different functions of the government. One dimension I'm hoping to do more work on is the study of environmental regulation – a big challenge that is tightly linked to implementation. There are federal environmental programs that are implemented in different places, with a lot of heterogeneity. You have that in the US, and there’s a similar structure in India, so there's a lot to be looked at!