How Google measures high-scoring management

Since the early days of Google, people throughout the company have questioned the value of managers.

As one software engineer, Eric Flatt, puts it, “We are a company built by engineers for engineers.” And most engineers want to spend their time designing and debugging, not communicating with bosses or supervising other workers’ progress.

A few years into the company’s life, founders Larry Page and Sergey Brin wondered whether Google needed any managers at all.

In 2002 they experimented with a completely flat organisation, eliminating engineering managers.

That experiment lasted only a few months. And as the company grew, the founders soon realized that managers contributed in many important ways.

Google now has some layers, but not as many as you might expect in an organisation with more than 37,000 employees: just 5,000 managers, 1,000 directors and 100 vice presidents.

Here’s the challenge Google faced: If your highly skilled hires don’t value management, how can you run the place effectively? How do you turn doubters into believers, persuading them to spend time managing others? You use data to test your assumptions about management’s merits and then make your case.

Analysing the soft stuff

In 2006 Page and Brin brought in Laszlo Bock to head up the human resources function — appropriately called people operations, or people ops. From the start, people ops managed performance reviews, which included annual 360-degree assessments. A year later, with that foundation in place, Bock hired Prasad Setty to lead a people analytics group. He challenged Setty to approach HR with the same empirical discipline Google applied to its business operations.

Setty recruited several Ph.D.s with serious research chops. In early 2009 people analytics presented its initial set of research questions to Setty. One question stood out: Do managers matter?

To find the answer, Google launched Project Oxygen, a multiyear research initiative. It has since grown into a comprehensive programme that measures key management behaviours and cultivates them through communication and training. By November 2012, the company had shown statistically significant improvements in multiple areas of managerial effectiveness.

Project Oxygen was designed to offer granular, hands-on guidance. It didn’t just identify desirable management traits in the abstract; it pinpointed specific, measurable behaviours that brought those traits to life.

That’s why Google employees let go of their skepticism and got with the programme. Data-driven cultures respond well to data-driven change.

Making the case

Project Oxygen co-lead Neal Patel and his team reviewed exit-interview data to see if employees cited management issues as a reason for leaving Google. Though they found some connections between turnover rates and low satisfaction with managers, those didn’t apply to the company more broadly, given the low turnover rates overall.

As a next step, Patel examined employee survey ratings and semi-annual reviews, comparing managers on both satisfaction and performance.

“At first,” he says, “the numbers were not encouraging. Even the low-scoring managers were doing pretty well. How could we find evidence that better management mattered when all managers seemed so similar?” The solution came from applying sophisticated multivariate statistical techniques, which showed that even “the smallest incremental increases in manager quality were quite powerful.”

The Project Oxygen team concluded that managers indeed mattered. But to act on that finding, Google first had to figure out what its best managers did. So the researchers followed up with double-blind qualitative interviews with high- and low-scoring managers. After much review, Oxygen identified eight behaviours shared by high-scoring managers.

Putting the findings into practice

The list of behaviours has served three important functions at Google: giving employees a shared vocabulary for discussing management, offering them straightforward guidelines for improving it and encapsulating the full range of management responsibilities.

It was clear early on that managers would need help adopting the new standards, so people ops built assessments and a training programme around the Oxygen findings.

People ops designed the training to be hands-on and immediately useful. In “vision” classes, for example, participants practised writing vision statements for their departments or teams and bringing the ideas to life with compelling stories. Managers have expressed few concerns about signing up for the training courses and going public with the changes they need to make.

To complement the training, the development team sets up panel discussions featuring high-scoring managers from each function. People ops also sends new managers automated email reminders with tips on how to succeed at Google and information about courses they haven’t taken.

And Google rewards the behaviours it’s working so hard to promote. The company has revamped its selection criteria for its Great Manager Award to reflect the eight Oxygen behaviours. The prize includes a weeklong trip to a destination such as Hawaii, where winners get to spend time with senior executives.

Measuring results

The people ops team has analysed Oxygen’s impact by examining aggregate survey data and qualitative input from individuals. The improvements were consistent across functions, survey categories, management levels, spans of control and geographic regions. Overall, managers took the feedback constructively.

Google executives speculated about the connection between employees’ performance reviews and their managers’ feedback scores. To address this question, the people analytics group fell back on a time-tested technique — going back to the data and conducting a formal analysis. The group found that changes in performance ratings accounted for less than 1 per cent of variability in corresponding manager ratings.

Project Oxygen does have its limits. A commitment to managerial excellence can be hard to maintain over the long haul. Sustainability depends on the continued effectiveness of managers who excel at the eight behaviours, as well as those behaviours’ relevance to senior executive positions.

Further, while survey scores gauge employees’ satisfaction and perceptions of the work environment, it’s unclear exactly what impact those intangibles have on such bottom-line measures as sales and profitability.

Still, Project Oxygen has accomplished what it set out to do: It not only convinced Googlers that managers mattered but also identified, described and institutionalised their most essential behaviours.

At a company like Google, where the staff consists almost entirely of “A” players, managers must provide steady feedback to guide people to greater levels of achievement — but intervene judiciously and with a light touch. When the process works well, it can yield extraordinary results.

That’s why Setty wants to keep building on Oxygen’s findings about effective management practice. His team has begun analysing managers’ assessment scores by personality type, looking for patterns.

And that, in a nutshell, is the principle at the heart of Google’s approach: deploying disciplined data collection and rigorous analysis to uncover deeper insights into the art and craft of management.

David A. Garvin is the C. Roland Christensen Professor of Business Administration at Harvard Business School.

© 2013 Harvard Business School Publishing Corp.

© The New York Times 2013