Archive for category learning
Promoting development
Posted by Theo in cognitive development, learning on October 4, 2009
There is a vast literature exploring ways to promote development. Much of this literature focuses on speeding up development, some of it focuses on optimizing development. Although both approaches are intended to support development, there is evidence that approaches focused on optimizing development are likely to do a better job. This is because development involves two intertwined processes, differentiation (broadening and deepening knowledge) and integration. In plain(er) English, you get more adequate integrations at each level if you accomplish rich differentiation at the prior level.
When we code an assessment, we pay close attention to the degree to which the test-taker elaborates each of the sub-skills it targets. In our personal feedback, we note areas of strength and areas that appear to require further growth. The basic idea is to bring all of the sub-skills up to an optimal level of elaboration to support the emergence of next-level integrations.
Most of the readings we suggest are targeted one to two phases (1/4 to 1/2 of a level) above the level of a given performance. This practice has been shown to provide the ideal level of challenge (scaffolding) for optimal growth. We also suggest activities like engaging in discourse with peers, journaling, cultivating a habit of reflection, and improving metacognitive skills, all of which provide support for growth.
We do not teach people to think at higher levels. Higher levels of performance emerge when knowledge is adequately elaborated and the environment supports higher levels of thinking and performance. We focus on helping people to think better at their current level and challenging them to elaborate their current knowledge and skills—including the not-so-sexy nuts-and-bolts knowledge required for success in any context.
Testing as part of learning 2
Posted by Theo in cognitive development, learning, motivation, teaching, testing on July 14, 2009
I can’t help it, I’m a developmental psychologist. I’ve been lurking about, watching my Granddaughter, Erwin, as she learns to master her environment. She’s about 8 months old now (real age, she was three months premature, so her birth age is 11 months)
Last week, Erwin figured out that complex actions can be used intentionally to make things happen in social situations. For example, she started reaching toward her Mom and Dad to indicate her intention to be picked up. At around same time, she began pointing to objects to indicate interest or draw them to the attention of her others. And she has begun to imitate actions like waving, clapping, and head shaking. Today, when we were Skyping, she clapped her hands to get me to play pat-a-cake, and she shakes her head to get her Mom to do the same—which she finds hilarious. To Mom’s dismay, Erwin is so excited by this new way of influencing her environment that she has stopped napping.
To see an example of Erwin’s attempts at verbal communication and her new reaching behavior, double-click on the picture below. Notice how emphatic her arm extension is, and how she makes eye contact as she reaches out.
A few months ago, most of Erwin’s actions were aimed toward physical mastery—learning to obtain ojects and manipulate them in a variety of ways, learning to move herself toward things she wanted to manipulate, or playing with sound just to hear the results.
When she was learning to do physical things, the physical environment provided most of the feedback. Although her parents were there to give encouragement, we all had the sense that it was the physical feedback that she craved—getting an object to her mouth, inching toward a favorite toy, pulling herself to stand.
Now she craves feedback from her parents; she has shifted her focus from physical mastery to social mastery. She reaches for Mom and gets picked up. She shakes her head and Mom shakes her head back. She points to a banana, and Dad brings it to her. She claps her hands, and Grandma plays pat-a-cake. And every time she undertakes a new action, she is conducting a test.
Testing is part of learning.
Each time any infant tries out a new skill, she is conducting a test. Each attempt is part of an action-feedback loop. Repeated attempts to master a new skill form a series of these action-feedback loops. Each iteration is an exemplary test—in the sense that it is educative—that guides the infant incrementally toward a new level of mastery.
Interestingly, infants never tire of this kind of testing, even when the feedback is not instantly gratifying. In fact, much of the feedback is along the lines of, “almost, but not quite,” or “that didn’t work,” neither of which seem to get in the way of infant learning. For example, when Erwin first started reaching toward her parents to ask to be picked up, her action was not easy to read. It rarely got the desired response. She gradually learned that the reaching needed to be clearly directed toward the parent and accompanied by eye contact. Now the message is, “You’ve got it!” At this point, Erwin takes the skill for granted, and has shifted her attention to things she has not yet mastered, like figuring out how to get adults to do other interesting or gratifying things.
The natural action-feedback mechanism of infancy works perfectly, because the proverbial carrot is usually, due to the very nature of normal human environments, dangled at just the right distance. Good parents respond to early attempts at communication, rewarding them with interesting responses, but success isn’t the only reward; it’s always accompanied by a new “carrot”—another interesting possibility just beyond the infant’s reach. In this way, the action-feedback mechanism functions both as an aid to learning and as a motivator.
Aspects of this “carrot-and-stick” perspective on learning have been expanded and described in a variety of research traditions—e.g., as part of the notion of reinforcement feedback in social learning theory (Bandura, 1977), as zone of proximal development in Vygotsky’s (1986) work, and as part of a complex process of assimilation and accommodation in Piaget’s (1985) work. It is important, because it speaks both to how we learn and to our motivation for learning. Good feedback plays two essential roles. First, it helps the learner decide what to try next. Second, it motivates the learner to keep striving toward mastery. And, as the infant example suggests, feedback cannot be reduced to simple reward or punishment. Ideally, it is information that supports learning by being useful to the learner. Learners are not motivated by reward or punishment per se, but by an optimal combination of “not there yet” “almost” and “you’ve got it”.
DiscoTests are for learning
Most of today’s tests provide feedback in the form of rewards (good grades, advancement, or honors) or punishment (bad grades and failure). My colleagues and I don’t find this acceptable, so we’ve created a nonprofit called DiscoTest. The overarching objective of the DiscoTest Initiative is to contribute to the development of optimal learning environments by creating assessments that deliver the kind of educative feedback that learners need to learn optimally. DiscoTests determine where students are in their individual learning trajectories and provide feedback that points toward the next incremental step toward mastery.
I’ll be writing more about DiscoTest in future posts. For now, if you’d like to know more, please visit the DiscoTest web site.
A good test
Posted by Theo in cognitive development, educational testing, learning, motivation, testing in general on April 29, 2009
In this post, I explore a way of thinking about testing that would lead to the design of tests that are very different from most of the tests students take today.
Two propositions, an observation, and a third proposition:
Proposition 1. Because adults who do not enjoy learning are at a severe disadvantage in a rapidly changing world, an educational system should do everything possible to nurture children’s inborn love of learning.
Proposition 2. In K-12, the specific content of a curriculum is not as important as the development of broadly applicable skills for learning, reasoning, communicating, and participating in a civil society. (The content of the curriculum would be chosen to support the development of these skills and could—perhaps should—differ from classroom to classroom.)
Observation. Testing tends to drive instruction.
Proposition 3. Consequently, tests should evaluate relevant skills and be employed in ways that support students’ natural love of learning.
Given these propositions, here is my favorite definition of a “good test.”
A good test is part of the conversation between a “student” and a “teacher” that tells the teacher what the student is most likely to benefit from learning next.
I’ll unpack this definition and show how it relates to the proposals listed above:
Anyone who has carefully observed an infant in pursuit of knowledge will understand the conversational nature of learning. A parent holds out a shiny spoon and an infant’s arms wave wildly. Her hand makes contact with the spoon and a message is sent to her brain, “Something interesting happened!” The next day, her arm movements are a little less random. She makes contact several times, feeling the same sense of satisfaction. Her parents laugh with delight. She coos. In this way, her physical and social environment provide immediate feedback each time she succeeds (or fails). Over time, the infant uses this information to learn how to reach out and touch the spoon at will. Of course, she is not satisfied with merely touching the spoon, and, through the same kind of trial and error, supplemented with a little support from Mom and Dad, she soon learns to bring the spoon to her mouth. And the conversation goes on.
Every attempt to touch the spoon is a kind of test. Every success is an affirmation that the strategy just employed was an effective strategy, but the story does not end here. In her quest to master her environment, the infant keeps moving the bar. Once she can do so at will, touching the spoon is no longer satisfying. She moves on to the next skill—holding the spoon, and the next—bringing it to her mouth, etc. Having observed this process hundreds of times, I strongly suspect that a sense of mastery is the intrinsic reward that motivates learning, while conversation, including both social and physical interactions, acts as the fuel.
Conversation
A good educational test should have the same quality of conversation, in the form of performance and feedback, that is illustrated in the example above. In an ideal testing situation, the student shows a teacher how he or she understands new concepts and skills, then the teacher uses this information to determine what comes next.
Part of the conversation
However, a good test is part of the conversation—not the entire conversation. No single test (or kind of conversation) will do. For example, the infant reaches for the spoon because she finds it interesting, and she must be interested enough to reach out many dozens of times before she can grasp an object at will. Good parents recognize that she expresses more sustained interest if they provide her with a number of different objects—and don’t try to force her to manipulate objects when she would rather be nursing or sleeping. Each act is a test embedded in a long conversation that is further embedded in a broader context.
What comes next?
In the story, I suggest that the spoon must be both interesting and within an infant’s reach before it can become part of an ongoing conversation. In the same way, a good test should both be engaging and within a student’s reach in order to play its role in the conversation between student and teacher.
An engaging test of appropriate skills can tell us how a student understands what he or she is learning, but this knowledge, by itself, does not tell the teacher (or the student) what comes next. To find out, researchers must study how particular concepts and skills are learned over time. Only when we have done a good job describing how particular skills and concepts are learned can we predict what a student is most likely to benefit from learning next.
So, a good test must not only capture the nature of a particular student’s understanding, it must also be connected to knowledge about the pathways through which students come to understand the concepts and skills of the knowledge area it targets.
Back to conversation
I argue above, that in infancy, a sense of mastery is the intrinsic reward that motivates learning, while conversation is the fuel. If conversation is the fuel, tests that do a good job serving the conversational function I outline here are likely to fuel students’ natural pursuit of mastery and a lifelong love of learning.
Later: But what about accountability?
Test validity & tacit knowledge
Posted by Theo in decision making, educational testing, learning, reasoning on April 21, 2009
As you probably know if you are reading this post, my colleagues and I make developmental assessments, several of which are focused on adult skills like managerial decision making. I am often asked about the validity of these assessments as it pertains to the distinction between intuitive or tacit knowledge and the kind of skills people need to do well on a developmental assessment. The short answer is that tacit knowledge is not captured by any assessment that asks people to reason through a problem, because tacit knowledge is, well, tacit.
How does it work?
Often, people know more about a particular subject than they can communicate verbally. This is because much of what we learn through experience does not automatically become part of our conscious knowledge. It’s in a form that’s difficult to share. We call this kind of knowledge tacit or implicit. You can also think of it as intuitive. Tacit knowledge is not a bad thing. It helps us make quick choices in familiar situations; we’d be in big trouble if we had to think through every single situation in our lives before we made a decision! But tacit knowledge has its limits.
First, because we aren’t able to bring it into focus, we can’t share tacit knowledge with others. This is the case in a business where the person at the top is an “intuitive” leader. Because his or her leadership skills are tacit, they can’t be shared. In situations like this, it is not uncommon for an institution to last only as long as its leader.
Second, tacit knowledge is limited by our direct experience, which means it isn’t terribly useful for dealing with novelty or abstractions. Because tacit knowledge is experienced-based, it is most useful in situations that are like those we have confronted in the past. Unfortunately for those who rely too heavily on tacit knowledge, the modern world constantly provides us with new challenges. To meet them, we need conscious methods for evaluating knowledge and experience.
Third, sometimes our tacit knowledge is sub-optimal. For example, people who learn the skills required to survive on the rough side of town or in an abusive relationship often fare poorly when they try to function outside of those contexts, partly because their tacit knowledge isn’t useful in their new surroundings. Before they can learn more adaptive skills, they usually need to bring their tacit knowledge into consciousness.
Finally, relying on tacit knowledge limits our development. When we habitually rely on our tacit knowledge, learning not only slows down, it’s quality is compromised. Optimal learning requires that we reflect consciously upon our own experiences and actively seek other perspectives to fill in the gaps.
Implications for assessment
The implications for assessment are clear. If knowledge is tacit, it won’t be reflected in an assessment of reasoning skills. Although good developmental assessments can provide accurate evaluations of the level of complexity a person can consciously cope with in a specific skill area, they can’t tell us everything we need to know about a person’s capabilities. This is one good reason—in a much longer list of reasons—to avoid making high stakes decisions on the basis of a single form of evaluation.
Testing as part of learning 1
Posted by Theo in educational testing, learning, testing in general on April 11, 2009
Learning isn’t easy
Yet all healthy babies pursue it with dogged determination, spending hour after hour exploring—and learning to master—their own bodies, as well as their physical and social environments.
Natural testing
When infants and young children engage their environments, they receive constant feedback about what does and does not work. For example, babies spend months learning how to control the movements of their hands. An infant will spend several weeks just learning how to bring an object to her mouth. She’ll use what she learns from successes and failures to do better next time. Feedback is instant and accurate, and the results of each attempt tell her what to try next.
Babies often act like they are addicted to learning. They will tolerate an amazing amount of failure. But without prompt feedback from their external environment, they wouldn’t get far. The same is true for older children.
Testing in schools
Ideally, educational tests model natural testing by providing students with timely and accurate feedback that tells them (and their teachers) what to try next.
Learning is boring
By the third grade, when students are asked to complete the sentence, “Learning is…”, their most common answer is boring.* Many adults would agree.
Does this mean that people fundamentally dislike learning? There are those who think so, but the evidence suggests otherwise. Babies, for example, absolutely love learning. You’d think it was their reason for being. They spend just about every waking moment engaged in activities that build knowledge and skills, and are so eager to learn that they press on despite repeated, often painful failures.
Of late, my granddaughter has reminded me of this inborn passion for learning. I’ve been fascinated by how hard she has practiced to learn to manipulate objects with her hands. At the beginning, she achieved almost no success. When a toy moved into range, she waved her arms wildly. Occasionally she would make contact, but these events were so random it was hard to see how she could learn from them. Apparently, however, she learned a little from each contact, because her rate of success improved almost daily. A few months later, she can pass a toy from one hand to another, taste it, hold it out to get a better look, and bang it on the table. All her parents had to do was provide an assortment of interesting objects. This was very easy, because Erwin, as is typical of babies, finds all objects interesting.
What happens between infancy and third grade? I think we have created an educational system (and a culture) that systematically undermines many of our our children’s inborn enthusiasm for the hard work and natural pleasures of learning.
I will elaborate in future posts, starting with some comments on the effects of America’s current obsession with standardized testing and something I call “bottom of the class” syndrome.
*If you teach, check it out. Write “Learning is…” on the blackboard and ask students to complete the sentence with one word (anonymously) on a sheet of paper. Collect and tabulate their answers. Have a discussion.
Falling in love with complexity
Posted by Theo in cognitive development, learning, reasoning on April 1, 2009
In college it’s a common thing. Students, exposed for the first time to the complex ideas of philosophers, psychologists, and scientists, fall in love with complexity. They want, more than anything, to be able to think like their heroes, and many assume that the best way to do this is to emulate their language. Big words, complex sentences, and abstract ideas—the more abstract the better—abound. Once competent writers temporarily lose their own voice. Students who have not already developed solid writing skills produce gobbledygook (complexity without integration). Good professors provide constructive feedback and support learning that will eventually tame the gobbledygook and unleash more mature voices.
Falling in love with complexity is now common in adults. Adult development is a popular idea these days, and many adults are jumping on the complexity bandwagon, which is fueled by an abundance of popular books that tout the superiority of systems thinking, dialectical reasoning, etc. What the authors of these books often don’t tell their readers is that learning to think in a more complex way usually requires years of study in a particular discipline*. Readers are led to believe that all they need to do is read a book, take a course, or learn to meditate and they will emerge as more complex, integrated and capable persons. Unfortunately, they often learn the language of complexity, but don’t do the nuts and bolts learning, doing, and reflecting that’s required for integration.
In our assessments, I frequently see evidence of this problem. Check out this example: A successful manager should be able to apply transformational leadership and various frameworks to gather, interpret, weigh, balance, and align different individual and institutional perspectives and interests, as well as psychological and sociological aspects to allow for the emergence of the successful determination of an ideal resolution process for this cultural context. Such an approach would require emotional intelligence and dialectics to determine a set of guiding principles that will lead to an introspective and interactive process of finding a solution that is appreciative of the interdependence and significance of all perspectives.
This is love.
*In adulthood, one full level of development on Fischer’s, Commons, Kegan’s, Kohlberg’s, Armon’s, Kitchener & King’s, Jaques’, or Fowler’s scales takes 4 to 10 years of continuous reflective study and practice. Another way to think about this is that it takes about 10,000 hours of reflective study and practice to become an expert. This is long enough to fall out of love with complexity and strike up a relationship with simplicity!
Test validity (part 1)
Posted by Theo in learning, standardized testing, testing in general on March 23, 2009
If a test is (1) measuring what it intends to measure (construct validity) and (2) what it is measuring is of value (ecological validity), it is considered to be a valid test. Sounds pretty straightforward, but it’s not. That’s partly because these two categories of validity often compete with one another, and it is a challenge to find the right balance.
For example, it seems pretty obvious that math items should be about math and reading comprehension items should be about reading comprehension. So, to make sure a math test has construct validity—is about math—you ought to limit the amount of reading required to understand your test items, right?
But what if what you really want to know is how students tackle real-world math problems, which often require the ability to understand the context in which mathematical problems are encountered. After all, there are good reasons to think that a skill a student can apply in real-world contexts is superior to a skill a student can only exhibit on a test that is stripped of context. If you followed this line of reasoning and composed your test of questions that reflect how knowledge is used in the world outside of the classroom, it would have ecological validity.
However, while including context in your math test would increase its ecological validity, doing so would increase the risk of reducing its construct validity by making it less clear exactly what is being measured. This might be reflected in lowered scores for students who can do math but aren’t good readers or are unfamiliar with the kind of situations described in test questions. A result like this can look a lot like discrimination—especially when the stakes are high.
In sum, the more you strip away context, the more you risk lowering ecological validity. The more context you add, the more you risk lowering construct validity. Today, there is a strong tendency to prioritize construct validity over ecological validity, primarily because the stakes of many tests are very high, which increases our focus on anything that seems to interfere with fairness. Without intending to, test developers, policy-makers, parents, and teachers have contributed to the creation of tests with decreasing ecological validity—and there is no doubt that teachers are teaching to these tests. The implication? What students are learning in our public schools is increasingly irrelevant.
This is a cause for concern.
Recent Comments