In my book on Organisation Culture I ask the question "Can culture be measured?" Here's an extract from the chapter that deals with that question.
Edgar Schein, in The Corporate Culture Survival Guide, discusses several reasons "why culture surveys do not and cannot measure culture":
1. There is no way of knowing what cultural dimensions are important in any one organisation. Even so, the surveys make an assumption that the dimensions they pick up are the same for all organisations that take that survey. In the example above the cultural dimensions being asked about focus on behaviour and include people's relationship to the organisation's mission, vision, and/or values: the "mood" of the organisation, the activities of leaders, what people complain about most, the structure of people's relationships. Without doing a lot more digging for information it is not clear on what basis these behaviours (if they are behaviours) have been picked out.
2. There is little recognition, at least in off the shelf surveys that culture covers all aspects of a group's external tasks and internal integration processes the ways these interact, integrate, and endlessly recombine. This is likely to be because these dynamic factors cannot be adequately addressed in a survey.
3. There is no way of knowing what a respondent is reading into the questions. In the example above, is 'mood' being experienced in the same way by everyone? Are complaints across the organisation the same and why are there only five things to complain about?
4. There is no way of knowing what the individual interpretation of a question is (or, indeed, if the respondent is answering honestly or in good faith), it is a leap of faith to assume that individual responses can be aggregated into a perspective on the whole organisation's culture. In this regard, Warren Bennis made the point that "you can't just pop a culture in a microwave and out pops a McCulture"
5. There is an implication that if the survey results show issues or problems then something will happen to address these. However, there may not be a will to do anything about the issues, for example if there are "clusters of people who feel they have very little impact on how the organisation is run", so what? If the organisation is performing well although lack of impact may be of importance to the responders it may not be a material finding to their managers. (Which, in itself, says something about the culture of the organisation).
Beyond the five reasons discussed by Schein there are several others that contribute to the case that surveys 'do not and cannot measure culture' including:
6. They take a reductionist view of culture which attempts to isolate independent cultural variables (norms, attitudes, values, behaviours, management practices, etc.) and 'treat' them. The problem with this is that even if surveys had meaningful variables they are not independent. Every variable is dependent in some way on every other. The previous chapter has made the argument that taking this approach is like taking an element of the weather (e.g. hours of sunshine in a given period) and assuming that manipulating this as an independent variable (were it possible) would change the weather in a desirable direction.
7. They imply that 'fixing' a culture that's 'broken' or working out what is great about a culture and needs protection can be quick and painless. Neither is the case. (Chapter 7 discusses the question 'Can culture be changed?').
8. They assume, by using the same dimensions in the survey, that organisation culture is homogeneous and shared across all employees. Again this is not typically the case. In organisations of any size there are sub-cultures, counter cultures, and variations in culture (as in the day to day weather analogy described in Chapter 1).
9. They take the view that organisation culture is 'transmitted' through behaviors and actions of employees within an organisation. This does not allow for the complexity of the interplay between internal and external 'transmission' routes and the chaotic and dynamic nature of culture.
10. They purport to be more than a snapshot in a particular time period - over-riding the dynamic and constantly changing nature of culture.
The popular Dilbert cartoon series created by Scott Adams has generated the 'Dilbert Metric' that neatly and absurdly makes the point about trying to measure an organisation's culture using surveys.
"The Dilbert metric involves showing ten Dilbert cartoons selected at random to members of an organisation. Each member rates the cartoons on a scale of zero to ten. Zero means the cartoon is wholly applicable to the organisation while ten means the cartoon is not at all relevant. Each individual's score is totalled up and an average score of all the totals is taken. The resulting figure is an indication of the health of the organisation's morale and culture: the closer the figure to zero, the closer the organisation to the Dilbertesque, chaotic view of corporate life; the closer to one hundred, the better".