Dear Crucial Skills,
In Crucial Conversations and Influencer you note studies that correlate changes in behavior to increases in productivity and/or profitability.
I will be going through Crucial Conversations trainer certification in January and would like to know how we can measure the impact of the classes after they are performed. What are some of the ways you’ve measured impact?
–Signed,
Results Oriented
Dear Results Oriented,
I was glad to get your question because it’s not one that we get too often: and that’s unfortunate.
The reason I’m so happy to get your question is that it’s a rare company that really pays the price to measure impact of organization development efforts. We cut our teeth on such activity and called ourselves VitalSmarts because we wanted to find the “vital skills” that profoundly and measurably affected crucial business results. It was that kind of measurement-oriented work that led us to discovering the principles in Crucial Conversations, Influencer, and our other published papers, books, and training programs.
Here are three tips for increasing the rigor of your influence work through measurement:
1. Something simple. If your goal is simply to improve the way your team functions, and the result you’d like to achieve is to improve communication about a few key issues within the team, you can gather useful data by creating a small survey and administering it yourself. For example, you and your teammates could agree on three to five different issues that you have a hard time discussing. Next, design survey items to measure the quality of your conversations around these few specific issues. Administer the survey before the training, immediately after the training, and again 60 and 180 days following. This approach with not give you hard evidence that you’ve affected organizational performance, but it will give you some indication about whether behavior in the team has changed.
2. A bit more rigorous. Something all three of these recommendations have in common is that the starting point is identifying the three to five crucial conversations that you and other leaders believe are integral to improving results. If you look at some of our studies (Silence Kills or Silence Fails) you’ll see how we have identified these crucial conversations.
If your goal is to test for relationships between improvements in crucial conversations and key results, you can take the first recommendation and simply track changes in the team’s results compared to improvement in survey results. While this is not hard statistical evidence, the comparison of improvements in team behavior to changes in bottom-line metrics will give you some reason to believe the two are related to each other.
3. Even deeper. Finally, you can go to the level of professional research to gather hard data that proves the correlation between training and results. We’ve used a variety of research methods in our work. All of the methods involve careful gathering and statistical analysis of data. For example, at Lockheed Martin we identified three crucial conversations we believed would influence improvement in quality, costs and employee engagement in the manufacturing area. We designed a survey to track improvement in these three crucial conversations. We then gathered data from 30 different teams in the factory from both behaviors (the crucial conversations survey) and results (quality, costs and employee engagement). When we ran statistical tests (both simple correlation and multiple regressions) we could demonstrate profound relationships between improvement in these three crucial conversations and results. Most importantly, by the simple act of drawing attention to both data sets, we helped leaders begin to see that there were literally no cases where teams substantially improved in results without also improving these three crucial conversations.
For those interested, I’d like to offer sample survey items you can use to get at various crucial conversations issues. These discussion questions are drawn from various client projects and are available for download in a PDF.
Good luck–and congratulations again on your desire to influence real and measurable change.
Warmly,
Joseph Grenny
