For several years now, one of our heroes, professor Howard Drossman of Colorado College and the Catamount Center, has been working with Lectical Assessments and helping us build LESA, the Lectical Environmental Stewardship Assessment.
Dr. Drossman's areas of expertise include developmental pedagogy, environmental stewardship, and the development of reflective judgment. His teaching focuses on building knowledge, skill, and passion through deep study, hands-on experience, and reflection.
For example, Dr. Drossman and ACM (Associated Colleges of the Midwest) offered a 10-day faculty seminar on interdisciplinary learning called Contested Spaces. This physically and intellectually challenging expeditionary learning experience provided participants with multiple disciplinary perspectives on current issues of land stewardship in the Pikes Peak region of Colorado.
A second, ongoing program is offered by Catamount Center and Colorado College is dedicated to inspiring the "next generation of ecological stewards." This program, called TREE (Teaching & Research in Environmental Education), is a 16-week, residential program for undergraduate students who have an interest in teaching and the environment. Program participants live and learn in community at the Catamount Mountain Campus, which is locatedin a montane forest outside of Woodland Park, Colorado. Through study and practice, they cultivate their own conceptions of environmental stewardship and respect for the natural world, while building skills for creating virtuous cycles of learning and useable knowledge in K-12 classrooms.
Dr. Drossman embeds Lectical Assessments in both of these programs, using them to customize instruction, support individual development, and measure program outcomes. He also is working closely with us on the development of the LESA, which is one of the first assessments we plan to bring online after our new platform, LecticaLive, has been completed.
We've argued for years that you can't really learn critical thinking by taking a critical thinking course. Critical thinking is a skill that develops through reflective practice (VCoL). Recently, a group of Stanford scientists reported that a reflective practice approach not only works in the short term, but it produces "sticky" results. Students who are routinely prompted to evaluate data get better at evaluating data—and keep evaluating it even after the prompts are removed.
Lectica is the only test developer that creates assessments that measure and support this kind of learning.
On November 20th, 2016, we presented a paper at the ACTFL conference in Boston. In this paper, we described the results of a 4-year research project, designed to address the question, "Does second language learning support the development of critical thinking as measured by the LRJA?". To learn more, view the presentation below.
There's a lot of talk about "embodied" learning these days, and it doesn't seem like there's much consensus about what it means. Since we sometimes use the term alongside "optimal learning" and "robust learning," I think it's time we offered a clear definition.
Take a close look the activity in the lesson shown above. I found it on the Shorewood School District's web site. The lesson depicted in this photo is an excellent example of embodied learning in action. Note the many ways in which students are engaged. They are trying to solve a problem: "What do we need to do to pick up this cup?" This problem has kinesthetic, mathematical, mechanical, and collaborative components. Minimally, the students are intellectually, physically, and socially engaged. And I'm sure they're emotionally engaged as well—I can practically feel their hearts beating faster as they get closer to their goal.
These children aren't just thinking about a solution, they're living the solution. What they learn is wired into their neural net at every level. It's not just an intellectual experience. It's embodied. This is what we call optimal or robust learning. It's the kind of learning we measure, support, and reward with Lectical Assessments.
Recently, members of our team at Lectica have been discussing potential misuses of Lectical Assessments, and exploring the possibility that they could harm some students. There are serious concerns that require careful consideration and discussion, and I urge readers to pitch in.
One of the potential problems we've discussed is the possiblilty that students will compare their scores with one another, and that students with lower scores will suffer from these comparisons. Here's my current take on this issue.
Students receive scores all the time. By third grade they already know their position in the class hierarchy, and live everyday with that reality. Moreover, despite the popular notion that all students can become above average if they work hard enough, average students don't often become above average students, which means that during their entire 12 years of schooling, they rarely receive top rewards (the best grades) for the hard work they do. In fact, they often feel like they're being punished even when they try their best. To make things worse, in our current system they're further punished by being forced to memorize content they haven't been prepared to understand, a problem that worsens year by year.
Lectica's approach to assessment can't prevent students from figuring out where their scores land in the class distribution, but we can give all students an opportunity to see themselves as successful learners, no matter where their scores are in that distribution. Average or below average students may still have to live with the reality that they grow at different rates than some of their peers, but they'll be rewarded for their efforts, just the same.
I've been told by some very good teachers that it is unacceptable to use the expression "average student." While I share the instinct to protect students from the harm that can come from labels, I don't share the belief that being an average student is a bad thing. Most of us were average students—or to be more precise, 68% of us were within one standard deviation of the mean. How did being a member of the majority become a bad thing? And what harm are we doing to students by creating the illusion that we are all capable of performing above the mean?
I don't think we hurt children by serving up reality. We hurt them when we mislead them by telling them they can all be above average, or when we make them feel hopeless by insisting that they all learn at the same pace, then punishing them when they can't keep up.
I'm not saying it's not possible to raise the average. We do it by meeting the specific learning needs of every student and making sure that learning time is spent learning robustly. But we can't change the fact that there's a distribution. And we shouldn't pretend this is the case.
Lectical Assessments are tests, and are subject to the same abuses as other tests. But they have three attributes that help mitigate these abuses. First, they allow all students without severe disabilities to see themselves as learners. Second, they help teachers customize instruction to meet the needs of each student, so more kids have a chance to achieve their full potential. And finally, they reward good pedagogy—even in cases in which the assessments are being misused. After all, testing drives instruction.
There are at least four reasons why people should learn robustly:
They'll learn more quickly.
They'll keep growing longer.
They'll be better prepared to participate fully in adult life.
Truly, there are no downsides to learning robustly. Yet robust learning is not what's happening for most students in most American schools. We have mounting—and disturbing—evidence that this is the case.
The data in the figure below are from our database of reflective judgment assessments. These are open-response formative assessments of how well people think about and address thorny real world problems like bullying, television violence, dietary practices, and global warming. We've been delivering these assessments for several years now and have a diverse sample of over 20,000 completed assessments to learn from.
We wanted to know how well schools are supporting development and what kind of role learning robustly might play in their performance. (Watch the video above to learn more about what counts as evidence of robust learning.) In particular, we wanted to know why students in one school—the Rainbow Community School—are outperforming students in other schools. (To learn about the Rainbow curriculum, click here.)
We first looked at one of the key sources of evidence for robust learning—the quality of students' arguments. In the figure below, the Y axis represents the quality or "coherence" of students' arguments and the X axis represents their Lectical phase (or developmental phase, 1/4 of a Lectical Level). The highest coherence score students can receive is a 10.
In this figure, the Rainbow Community School is the clear leader, especially when it comes to students performing in lower phases, with inner-city (primarily low socioeconomic status) public schools at the low end, and more conventional private schools and high socioeconomic status public schools in the middle. So, how does this relate to student development? Since we regard coherence of argumentation as strong evidence of robust learning, and assert that robust learning is required to support optimal development, we would expect Rainbow students to develop more rapidly than students in schools with lower coherence scores.
The figure below tells the story. When it comes to students' development on the Lectical Scale, Rainbow Community School students are way ahead of the pack. And our inner city schools are way behind. In fact, the average senior in our large (over 10,000 assessments) inner city sample is 5 years behind the projected score for the average senior in the Rainbow sample. Or in other words, inner city seniors, on average, are performing at the same level as Rainbow 7th graders.
We know socioeconomic status is a factor that contributes to this gap, but shouldn't our schools be closing it rather than allowing it to grow larger? Take a look at the figure below. This figure assumes that students in the Rainbow Community School, on average, start out at about the same developmental level as students in private and high SES public schools, yet student growth is faster. In fact, the data project that Rainbow 9th graders would perform as well as seniors in the other schools. That's a 3-year advantage! We believe this difference is due to differences in instructional practices. What if we used these same practices in our inner city schools? If we could accelerate their learning as much as the Rainbow Community School has accelerated the learning of its students, inner-city students would be doing as well as private and high SES public schools!
Although socioeconomic status is a key factor, we think the differences seen here are at least partially due to fundamentally different ways of thinking about learning and teaching. Conventional schools tend to be primarily content focused. There is an emphasis on learning as remembering. The Rainbow Community School is skill focused. Its teachers use content as a vehicle for building core life skills, such as skills for learning, inquiry, evaluating information, making connections, communicating, conflict resolution, decision making, mindfulness, compassion, and building relationships. To build these skills students continuously engage in virtuous cycles of learning—cycles of information gathering, application, reflection, and goal setting—that exercise these skills while building robust connections between new and existing knowledge. Students not only learn content, they learn to use it effectively in their everyday lives. It becomes part of them. We call this embodied learning.
We're eager to study the impact of skill-focused curricula on the learning of less advantaged students. If you know of a school that's fostering robust learning AND serving disadvantaged students, we'd like to help them show off what they're accomplishing.
Note: Not only does Rainbow Community School ensure that its students are continuously engaged in VCoLs (virtuous cycles of learning), it uses a system of governance, Sociocracy, that supports virtuous cycling for everyone on staff as well as the continuous improvement of its curriculum.
Appendix: Sample responses from 8th graders in different schools
Examples are taken from performances of students with average scores for their school.
The question students answered: How is it possible that the two groups [pro and anti bullying] have such different ideas?
Rainbow Community School
It could be due to different experiences. Perhaps the ones going for the argument that a little bullying can be okay were disciplined more at home and have a tougher shell for things like this. [Parents] may base their initial ideas on their own experiences or their children's. It all really depends on the person and how they were raised.
High SES public School
This because they have different ideas and reasons for thinking what they believe and you can't change that. The parents are not the same and every one of them is different so they have a right to believe what they want to believe.
Low SES public school
Many people think different and many people look at things differently. So people get different ideas and opinions about things.
The Goldilocks Zone, as we define it, builds upon a century of behavioral and brain research, starting with the zone of proximal development (ZPD).
Before we get to the ZPD, l'll describe the natural learning cycle as we see it today. It's a cycle that involves setting a learning goal, making an effort toward that goal, receiving feedback, then setting a new goal based on that feedback. The "fuel" that keeps the cycle going consists of two kinds of chemicals released by the brain—opioids, which make you feel great when you acheive a goal, and dopamine, which makes you want to strive for more successes. When the learning goal is just difficult enough to pose a challenge and not so easy you succeed every time—just right—the cycle will keep operating indefinitely. I've added a video to the bottom of this post that explains this process in more detail.
In the early part of the 20th century, Lev Vygotsky, a Russian psychologist and educator, identified what he called “the zone of proximal development.” This is the zone that lies just beyond an individual’s current level of skill. By identifying this zone accurately, Vygotsky and his colleagues were able to help children learn more effectively—especially the most mentally disadvantaged.
Then, Albert Bandura discovered that tasks that are “just challenging enough” are more engaging and support more effective learning. And more recently, brain scientists have learned that experiencing an optimal level of challenge relative to success supports our brain’s motivational system via the dopamine (wanting) opioid (pleasure) cycle, and Aime Stahl, has added more fuel to the fire with her finding that surprising events—events that set up dissonance—appear to prime the brain for learning.
The evidence is in. We can't afford to ignore the motivational power of creating the amount of dissonance, challenge, or surprise that is “just right” for an individual in a particular learning context at a particular moment in time. Lectical Assessments are designed to help educators zoom in on the zone.
What happens if you consistently punish someone for engaging in a particular practice? Most of us assume that it will get them to stop. Right? So, why have we developed an educational system that punishes millions of children for learning?
Because all capabilities are distributed in the population as “bell curves,” half of all children inevitably will be in the bottom half of their class. Most of these students will consistently receive grades that reflect poor performance relative to other students, primarily because (for a variety of reasons) they learn more slowly than their age mates. Young students tend to understand poor grades as punishments for poor performance or evidence of stupidity. The occasional low grade that is clearly attached to a lack of effort can act as an incentive to try harder. But consistently low grades with no hope of improvement teach students that learning is bad, because no matter how hard they try, it leads to punishment. I call this “bottom of the class syndrome.”
Clearly “learning is bad” is not the message teachers mean to send. They expect the punishment of a low grade to motivate students and their parents to try harder, and this is likely to work for many of the students in the upper half of the class. But it is not likely to work for kids in the bottom half of the class, because most of them are consistently slower learners. Most students in the bottom half of the class need more time and must make more effort to learn concepts and skills than students in the upper half of the class. They can’t afford to dislike learning; even more than students in the top of the class, they must retain their inborn love of learning to have any hope of success over the long term.
As long as we award scores that rank students, about half of those students will be vulnerable to “bottom of the class syndrome.” Society will lose many of them as learners, and their life choices and contributions will be unnecessarily restricted.
My colleagues and I have been working on this problem for many years. The solution we offer requires sophisticated ways of thinking about learning and motivation as well as new tools for evaluating learning—tools that don’t compare students, but allow each of them to develop on their own timeline. And it requires that we systematically study how students learn every single skill or concept we teach, so we know what every score on every assessment really means—in terms of what a given student understands and what he or she is likely to benefit from learning next. It’s hard work, but we’ve learned how to do it. Humans have tackled much more challenging problems. All we need is the will to ensure that every child has an opportunity to realize his or her potential, the patience to do the work, good people to carry the work forward, and a little thing called funding.
Ideal learning occurs in virtuous cycles—repeating cycles of goal setting, observation (taking in new knowledge), testing (applying what has been learned and getting feedback on results), and reflection (figuring out which adjustments are needed to improve one’s performance on the next attempt). This process, which occurs unconsciously from birth, can be made conscious. One recent application of the virtuous cycle is in dynamic steering, in which decisions are developed, applied, and evaluated through intentionally iterating cycles. The idea is to stretch as far as possible within a given cycle, without setting immediate goals that are completely beyond one’s reach. Success emerges from the achievement of a series of incremental goals, each of which brings one closer to the final goal. Processes of this kind lay down foundational skills that support resilience and agility. For example, the infant who learns to walk also learns to fall more gracefully, which makes learning to run much less traumatic than it might have been. And decision makers who use dynamic steering learn a great deal about what makes decisions more likely to be successful, which leads to better, faster decisions in increasingly complex or thorny situations.
The figure on the right illustrates how educators can support virtuous learning cycles. There are 4 "steps" in this process (not necessarily in the following order):
Find out what individual learners already know and how they work with their knowledge, then set provisional learning goals.
Acquire and evaluate information
Apply new knowledge or skills in hypothetical or real-life situations.
Provide frequent opportunities for learners to reflect upon outcomes associated with the application of new knowledge in an environment in which ongoing learning, application, and reflection are consistently rewarded.
At the end of October, the Century Foundation released a paper entitled, Eight reasons not to tie teacher pay to standardized test results. I agree with their conclusions, and would add that even if all standardized tests were extremely reliable and measured exactly what they intended to measure, this would be a bad idea. This is because success in the adult world requires a multiplicity of skills and forms of knowledge, and tests focus on only some of these, one at a time. Until we can construct multifaceted longitudinal stories about the progress of individual students that are tied to a non-arbitrary standardized metric, we should not even consider linking student evaluations to teacher pay.