Archive for category decision making

Task demands and capabilities

Our developmental assessment system, called the Lectical Assessment System (LAS), can be used to score (a) the performances of persons and (b) the task demands of specific situations/contexts. For example, my colleagues and I have analyzed the task demands of levels of management in large organizations, and tested managers’ developmental level of performance in several skill areas—including reasoning about leadership, reflective judgment, and decision-making.

The figure above shows the relation between the task demands of 7 levels of management and the performance levels of managers occupying these management positions. In this oversimplified image, the task demands of most management positions increase in a linear fashion, spanning levels 10-13. The capabilities of managers do not, for the most part, match these task demands.

This pattern is pervasive—we see it everywhere we look—and it reflects a hard truth. None of us is capable of meeting the task demands of the most complex situations in today’s world. I’ve come to believe that in many situations our best hope for meeting these demands is to (1) work strategically on the development of our own skills and knowledge, (2) learn to work closely with others who represent a wide range of perspectives and areas of expertise, and (3) use the best tools available to scaffold our thinking.

, ,

4 Comments

Test validity & tacit knowledge

As you probably know if you are reading this post, my colleagues and I make developmental assessments, several of which are focused on adult skills like managerial decision making. I am often asked about the validity of these assessments as it pertains to the distinction between intuitive or tacit knowledge and the kind of skills people need to do well on a developmental assessment. The short answer is that tacit knowledge is not captured by any assessment that asks people to reason through a problem, because tacit knowledge is, well, tacit.

How does it work?

Often, people know more about a particular subject than they can communicate verbally. This is because much of what we learn through experience does not automatically become part of our conscious knowledge. It’s in a form that’s difficult to share. We call this kind of knowledge tacit or implicit. You can also think of it as intuitive. Tacit knowledge is not a bad thing. It helps us make quick choices in familiar situations; we’d be in big trouble if we had to think through every single situation in our lives before we made a decision! But tacit knowledge has its limits.

First, because we aren’t able to bring it into focus, we can’t share tacit knowledge with others. This is the case in a business where the person at the top is an “intuitive” leader. Because his or her leadership skills are tacit, they can’t be shared. In situations like this, it is not uncommon for an institution to last only as long as its leader.

Second, tacit knowledge is limited by our direct experience, which means it isn’t terribly useful for dealing with novelty or abstractions. Because tacit knowledge is experienced-based, it is most useful in situations that are like those we have confronted in the past. Unfortunately for those who rely too heavily on tacit knowledge, the modern world constantly provides us with new challenges. To meet them, we need conscious methods for evaluating knowledge and experience.

Third, sometimes our tacit knowledge is sub-optimal. For example, people who learn the skills required to survive on the rough side of town or in an abusive relationship often fare poorly when they try to function outside of those contexts, partly because their tacit knowledge isn’t useful in their new surroundings. Before they can learn more adaptive skills, they usually need to bring their tacit knowledge into consciousness.

Finally, relying on tacit knowledge limits our development. When we habitually rely on our tacit knowledge, learning not only slows down, it’s quality is compromised. Optimal learning requires that we reflect consciously upon our own experiences and actively seek other perspectives to fill in the gaps.

Implications for assessment

The implications for assessment are clear. If knowledge is tacit, it won’t be reflected in an assessment of reasoning skills. Although good developmental assessments can provide accurate evaluations of the level of complexity a person can consciously cope with in a specific skill area, they can’t tell us everything we need to know about a person’s capabilities. This is one good reason—in a much longer list of reasons—to avoid making high stakes decisions on the basis of a single form of evaluation.

, ,

No Comments