National leaders’ thinking: What we’ve learned so far…

In this article, I’ll be providing a summary of results from each group of leaders observed as part of Lectica’s National Leaders’ Study. Each time my colleagues and I complete a round of research for a particular group of national leaders, the results will first be presented in a special article, then summarized here. This article will be written and rewritten over several months, with regular updates. If at any point you want to get a quick sense of what we’ve learned so far, just come back to this article for an overview.

Summary of quantitative results

The following table compares the scores received by the leaders of countries included in the National Leaders’ Study so far. (If you don’t yet know what I mean by complexity level, see the first article in this series.

Country

Complexity score range

Complexity score difference

Leader average

Media average

Leader average – media average

USA

1054–1163

109

1116

1137 (without P. Trump

1124

-8

13

Australia

1111–1133

22

1125

1111

14

Key observations

  1. Lowest score: The average complexity level of President Trump’s interviews was 1054—near the average score received by 12th graders in a good high school.
  2. Highest score: The mean score for President Obama’s first two interviews was 1193. This is well above the average score received by CEOs in Lectica’s database and is in the ideal range for a national leader, who must be able to comprehend and work with issues that have a complexity level of 1200 and above.
  3. Fit-to-role: With the exception of Barack Obama, none of the leaders so far has demonstrated (in their interviews)  a level of complexity that is a good match for the complexity level of many of the problems faced in office (1200+).
  4. Third interview scores: The scores of three out of 5 leaders whose scores at time 1 were above the level of average media scores—Barack Obama, Tony Abbott, and Malcolm Turnbull—dropped closer to media averages in their third interviews. We’re monitoring this potential trend.
  5. Media score comparison: The mean score for sampled U. S. media was 13 points higher than the mean score for Australian media.
  6. Leader score comparison: If we exclude President Trump as an extreme outlier, the average score for U. S. Presidents was 9 points higher than the average score for Australian prime ministers.

Emerging concerns

  1. Difficulty evaluating candidates: In the interest of accessibility, voters are systematically being deprived of the evidence required to evaluate the competence of candidates. High-profile interview responses of national leaders are often the only place to observe anything like the actual thinking of candidates for office, yet it is well known that candidates and leaders are trained to simplify responses to interview questions. Moreover, national leaders’ speeches are written in language that simplifies issues to make them more accessible to the general public, and many candidates have not produced written works that can be relied upon as evidence of current capacity.
  2. Danger of electing incompetent candidates: When all candidates produce responses and read speeches in which issues are systematically simplified, it becomes very difficult to distinguish between different candidates’ level of understanding. This makes it easier to elect candidates that lack the level of understanding and skill required to cope with highly complex national and international issues.

Other articles in this series

Please follow and like us:

National Leaders’ thinking: Australian Prime Ministers

How complex are the interview responses of the last four Australian prime ministers? How does the complexity of their responses compare to the complexity of the U.S. presidents’ responses?

Special thanks to my Australian colleague, Aiden M. A. Thornton, PhD. Cand., for his editorial and research assistance.

This is the 4th in a series of articles on the complexity of national leaders’ thinking, as measured with CLAS, a newly validated electronic developmental scoring system. This article will make more sense if you begin with the first article in the series.

Just in case you choose not to read or revisit the first article, here are a few things to keep in mind:

  • I am an educational researcher and the CEO of a nonprofit that specializes in measuring the complexity level of people’s thinking skills and supporting the development of their capacity to work with complexity.
  • The complexity level of leaders’ thinking is one of the strongest predictors of leader advancement and success. See the National Leaders Intro for evidence.
  • Many of the issues faced by national leaders require principles thinking (level 12 on the skill scale/LecticalScale), illustrated in the figure below). See the National Leaders Intro for the rationale.
  • To accurately measure the complexity level of someone’s thinking (on a given topic), we need examples of their best thinking. In this case, that kind of evidence wasn’t available. As an alternative, my colleagues and I have chosen to examine the complexity level of prime ministers’ responses to interviews with prominent journalists.

Benchmarks for complexity scores

  • Most high school graduates perform somewhere in the middle of level 10.
  • The average complexity score of American adults is in the upper end of level 10, somewhere in the range of 1050–1080.
  • The average complexity score for senior leaders in large corporations or government institutions is in the upper end of level 11, in the range of 1150–1180.
  • The average complexity score (reported in our National Leaders Study) for the three U. S. presidents that preceded President Trump was 1137.
  • The average complexity score (reported in our National Leaders Study) for President Trump was 1053.
  • The difference between 1053 and 1137 generally represents a decade or more of sustained learning. (If you’re a new reader and don’t yet know what a complexity level is, check out the National Leaders’ Series introductory article.)

The data

In this article, we examine the thinking of the four most recent prime ministers of Australia—Julia Gillard, Kevin Rudd, Tony Abbott, and Malcolm Turnbull. For each prime minister, we selected 3 interviews, based on the following criteria: They

  1. were conducted by prominent journalists representing respected news media;
  2. included questions that requested explanations of the Prime Minister’s perspective; and
  3. were either conducted within the Prime Minister’s first year in office or were the earliest interviews we could locate that met the first two criteria.

As noted in the introductory article of this series, we do not imagine that the responses provided in these interviews necessarily represent competence. It is common knowledge* that prime ministers and other leaders typically attempt to tailor messages to their audiences, so even when responding to interview questions, they may not show off their own best thinking. Media also tailor writing for their audiences, so to get a sense of what a typical complexity level target for top media might be, we used CLAS to score 11 articles from Australian news media on topics similar to those discussed by the four presidents in their interviews. We selected these articles at random—literally selecting the first ones that came to hand—from recent issues of the Canberra Times, The Age, the Sydney Morning Herald, and Adelaide Now. Articles from all of these newspapers landed in the lower range of the early systems thinking zone, with a mean score of 1109 (15 points lower than the mean for the U.S. media sample) and a range of 45 points.

Hypothesis

Based on the mean media score, and understanding that politicians generally attempt, like media, to tailor messages for their audience, we hypothesized that prime ministers would aim for a similar range. Since the mean score for the Australian media sample was lower by 15 points than the mean score for the U. S. media sample, we anticipated that the average score received by Australian prime ministers would be a bit lower than the average score received by U. S. presidents.

The results

The Table below shows the complexity scores received by the four prime ministers. (Contact us if you would like a copy of the interviews.) Complexity level scores are shown in the same order as interview listings.

All of the scores received by Australian prime ministers fell well below the complexity level of many of the problems faced by national leaders. Although we cannot assume that the interview responses we scored are representative of these leaders’ best thinking, we can assert that we can see no evidence in these interviews that these prime ministers have the capacity to grasp the full complexity of many of the issues they faced (or are currently facing) in office. Instead, their scores suggest levels of skill that are more appropriate for mid- to upper-level managers in large organizations.

Prime minister

Interview by date

Complexity level scores

Mean complexity level

Mean zone

Julia Gillard (2010-2013)

Laurie Oakes, Weekend Today, 6/27/2010; Jon Faine, ABC 774, 6/29/2010; Deborah Cameron, ABC Sydney, 7/07/2010

1108, 1113, 1113

1111

Early systems thinking

Kevin Rudd (2013-2013)

Kerry O’Brien, ABC AM, 4/24/2008; Lyndal Curtis, ABC AM, 5/30/2008; Jon Faine, ABC 774 Brisbane, 6/06/2008

1133, 1138, 1129

1133

Early systems thinking

Tony Abbott (2013-2015)

Alison Carabine, ABC Radio National, 12/16/2013; Ray Hadley, 1/29/2014; Chris Uhlman, ABC AM, 9/26/2014

1133, 1129, 1117

1126

Early systems thinking

Malcolm Turnbull (2015-)

Michael Brissendon, ABC AM, 9/21/2015; Several journalists, 12/1/2015; Steve Austin, ABC Radio Brisbane, 1/17/2017

1133, 1138, 1113

1128

Early systems thinking

Comparison of U.S. and Australian results

There was less variation in the complexity scores of Australian prime ministers than in the complexity scores of U. S. presidents. Mean scores for the U. S. presidents ranged from 1054–1163 (109 points), whereas the range for Australian prime ministers was 1111–1133 (22 points). If we exclude President Trump as an extreme outlier, the mean score for U. S. Presidents was 12 points higher than for Australian prime ministers.

You may notice that the scores of two of the prime ministers who received a score of 1133 on their first interview, had dropped by the time of their third interview. This is reminiscent of the pattern we observed for President Obama.

The mean score for all four prime ministers was 14 points higher than the mean for sampled media. Interestingly, if we exclude President Trump as an extreme outlier, the difference between the average score received by U. S. presidents is almost identical at 13 points. Almost all of the difference between the mean scores of prime ministers and presidents (excluding President Trump) could be explained by media scores.

Country

Complexity score range

Complexity score difference

Leader average

Media average

Leader average – media average

USA

1054–1163

109

1116

1137 (without P. Trump

1124

-8

13

Australia

1111–1133

22

1125

1111

14

The sample sizes here are too small to support a statistical analysis, but once we have conducted our analyses of the British and Canadian prime ministers, we will be able to examine these trends statistically—and find out if they look like more than a coincidence.

Discussion

In the first article of this series, I discussed the importance of attempting to “hire” leaders whose complexity level scores are a good match for the complexity level of the issues they face in their roles. I then posed two questions:

  • When asked by prominent journalists to explain their positions on complex issues, what is the average complexity level of national leaders’ responses?
  • How does the complexity level of national leaders’ responses relate to the complexity of the issues they discuss?”

We now have a third question to add:

  • What is the relation between the complexity level of National Leaders’ interview responses and the complexity level of respected media?

So far, we have learned that when national leaders explain their positions on complex issues, they do not — with the possible exception of President Obama — demonstrate that they are capable of grasping the full complexity of these issues. On average, their explanations do not rise to the mean level demonstrated by executive leaders in Lectica’s database.

We have also learned that when national leaders explained their positions on complex issues to the press, their explanations were 13–14 points higher on the Lectical Scale than the average complexity level of sampled media articles. We will be following this possible trend in upcoming articles about the British and Canadian leaders.

Interestingly, the Lectical Scores of two prime ministers whose average scores were above the media average dropped closer to the media average in their third interviews. We observed the same pattern for President Obama. It’s too soon to declare this to be a trend, but we’ll be watching.

As noted in the article about the thinking of U. S. presidents, the world needs leaders who understand and can work with highly complex issues, and particularly in democracies, we also need leaders whose messages are accessible to the general public. Unfortunately, the drive toward accessibility seems to have led to a situation in which candidates are persuaded to simplify their messages, leaving voters with one less way to evaluate the competence of our future leaders. How are we to differentiate between candidates whose capacity to comprehend complex issues is only as complex as that of a mid-level manager and candidates who have a high capacity to comprehend and work with these issues but feel compelled to simplify their messages? And in a world in which people increasingly seem to believe that one opinion is as good as any other, how do we convince voters of the critical importance of complex thinking and the expertise it represents?


*The speeches of presidents are generally written to be accessible to a middle school audience. The metrics used to determine reading level are not measures of complexity level. They are measures of sentence, word length, and sometimes the commonness of words. For more on reading level see: How to interpret reading level scores.


 Other articles in this series

Please follow and like us:

President Trump passed the Montreal Cognitive Assessment

Shortly after the President passed the Montreal Cognitive Assessment, a reader emailed with two questions:

  1. Does this mean that the President has the cognitive capacity required of a national leader?
  2. How does a score on this test relate to the complexity level scores you have been describing in recent posts?

Question 1

A high score on the Montreal Cognitive Assessment dos not mean that the President has the cognitive capacity required of a national leader. This test result simply means there is a high probability that the President is not suffering from mild cognitive impairment. (The test has been shown to detect existing cognitive impairment 88% of the time [1].) In order to determine if the President has the mental capacity to understand the complex issues he faces as a National Leader, we need to know how complexly he thinks about those issues.

Question 2

The answer to the second question is that there is little relation between scores on the Montreal Cognitive Assessment and the complexity level of a person’s thinking. A test like the Montreal Cognitive Assessment does not require the kind of thinking a President needs to understand highly complex issues like climate change or the economy. Teenagers can easily pass this test.

Related articles


Benchmarks for complexity scores

  • Most high school graduates perform somewhere in the middle of level 10.
  • The average complexity score of American adults is in the upper end of level 10, somewhere in the range of 1050–1080.
  • The average complexity score for senior leaders in large corporations or government institutions is in the upper end of level 11, in the range of 1150–1180.
  • The average complexity score (reported in our National Leaders Study) for the three U. S. presidents that preceded President Trump was 1137.
  • The average complexity score (reported in our National Leaders Study) for President Trump was 1053.
  • The difference between 1053 and 1137 generally represents a decade or more of sustained learning. (If you’re a new reader and don’t yet know what a complexity level is, check out the National Leaders Series introductory article.)

[1] JAMA Intern Med. 2015 Sep;175(9):1450-8. doi: 10.1001/jamainternmed.2015.2152. Cognitive Tests to Detect Dementia: A Systematic Review and Meta-analysis. Tsoi KK, Chan JY, Hirai HW, Wong SY, Kwok TC.

 

Please follow and like us:

President Trump on climate change

How complex are the ideas about climate change expressed in President Trump’s tweets? The answer is, they are even less complex than ideas he has expressed about intelligence, international trade, and immigration—landing squarely in level 10. (See the benchmarks, below, to learn more about what it means to perform in level 10.)

The President’s climate change tweets

It snowed over 4 inches this past weekend in New York City. It is still October. So much for Global Warming.
2:43 PM – Nov 1, 2011

 

It’s freezing in New York—where the hell is global warming?
2:37 PM – Apr 23, 2013

 

Record low temperatures and massive amounts of snow. Where the hell is GLOBAL WARMING?
11:23 PM – Feb 14, 2015

 

In the East, it could be the COLDEST New Year’s Eve on record. Perhaps we could use a little bit of that good old Global Warming…!
7:01 PM – Dec 28, 2017

Analysis

In all of these tweets President Trump appears to assume that unusually cold weather is proof that climate change (a.k.a., global warming) is not real. The argument is an example of simple level 10, linear causal logic that can be represented as an “if,then” statement. “If the temperature right now is unusually low, then global warming isn’t happening.” Moreover, in these comments the President relies exclusively on immediate (proximal) evidence, “It’s unusually cold outside.” We see the same use of immediate evidence when climate change believers claim that a warm weather event is proof that climate change is real.

Let’s use some examples of students’ reasoning to get a fix on the complexity level of President Trump’s tweets. Here is a statement from an 11th grade student who took our assessment of environmental stewardship (complexity score = 1025):

“I do think that humans are adding [gases] to the air, causing climate change, because of everything around us. The polar ice caps are melting.”

The argument is an example of simple level 10, linear causal logic that can be represented as an “if,then” statement. “If the polar ice caps are melting, then global warming is real.” There is a difference between this argument and President Trump’s argument, however. The student is describing a trend rather than a single event.

Here is an argument made by an advanced 5th grader (complexity score = 1013):

“I think that fumes, coals, and gasses we use for things such as cars…cause global warming. I think this because all the heat and smoke is making the years warmer and warmer.”

This argument is also an example of simple level 10, linear causal logic that can be represented as an “if,then” statement. “If the years are getting warmer and warmer, then global warming is real.” Again, the difference between this argument and President Trump’s argument is that the student is describing a trend rather than a single event.

I offer one more example, this time of a 12th grade student making a somewhat more complex argument (complexity score = 1035).

“The temperature has increased over the years and studies show that the ice is melting in the north and south pole, so, yes humans are causing climate change.”

This argument is also an example of level 10, linear causal logic that can be represented as an “if,then” statement. “If the temperature has increased and studies show that the ice at the north and south poles are melting, then humans are causing climate change. But in this case, the student has mentioned two trends (warming and melting) and explicitly uses scientific evidence to support her conclusion.

Based on these comparisons, it seems clear that President Trump’s Tweets about climate change represent reasoning at the lower end of level 10.

“Humans have caused a lot of green house gasses…and these have caused global warming. The temperature has increased over the years and studies show that the ice is melting in the north and south pole, so, yes humans are causing climate change.

This argument is also an example of level 10, linear causal logic that can be represented as an “if,then” statement. “If the temperature has increased and studies show that the ice at the north and south poles are melting, then humans are causing climate change. In this case, the student’s argument is a bit more complex than in previous examples. She has mentioned two variables (warming and melting) and explicitly uses scientific evidence to support her conclusion.

Based on these comparisons, it seems clear that President Trump’s Tweets about climate change represent reasoning at the lower end of level 10.

Reasoning in level 11

Individuals performing in level 11 recognize that climate is an enormously complex phenomenon that involves many interacting variables. They understand that any single event or trend may be part of the bigger story, but is not, on its own, evidence for or against climate change.

Summing up

It concerns me greatly that someone who does not demonstrate any understanding of the complexity of climate is in a position to make major decisions related to climate change.


Benchmarks for complexity scores

  • Most high school graduates perform somewhere in the middle of level 10.
  • The average complexity score of American adults is in the upper end of level 10, somewhere in the range of 1050–1080.
  • The average complexity score for senior leaders in large corporations or government institutions is in the upper end of level 11, in the range of 1150–1180.
  • The average complexity score (reported in our National Leaders Study) for the three U. S. presidents that preceded President Trump was 1137.
  • The average complexity score (reported in our National Leaders Study) for President Trump was 1053.
  • The difference between 1053 and 1137 generally represents a decade or more of sustained learning. (If you’re a new reader and don’t yet know what a complexity level is, check out the National Leaders Series introductory article.)

 

Please follow and like us:

President Trump on intelligence

How complex are the ideas about intelligence expressed in President Trump’s tweets?

President Trump recently tweeted about his intelligence. The media has already had quite a bit to say about these tweets. So, if you’re suffering from Trump tweet trauma this may not be the article for you.

But you might want to hang around if you’re interested in looking at these tweets from a different angle. I thought it would be interesting to examine their complexity level, and consider what they suggest about the President’s conception of intelligence.

In the National Leaders Study, we’ve been using CLAS — Lectica, Inc.’s electronic developmental scoring system—to score the complexity level of several national leaders’ responses to questions posed by respected journalists. Unfortunately, I can’t use CLAS to score tweets. They’re too short. Instead, I’m going to use the Lectical Dictionary to examine the complexity of ideas being expressed in them.


If you aren’t familiar with the National Leaders series, you may find this article a bit difficult to follow.


The Lectical Dictionary is a developmentally curated list of about 200,000 words or short phrases (terms) that represent particular meanings. (The dictionary does not include entries for people, places, or physical things.) Each term in the dictionary has been assigned to one of 30 developmental phases, based on its least complex possible meaning. The 30 developmental phases span first speech (in infancy) to the highest adult developmental phase Lectica has observed in human performance. Each phase represents 1/4 a level (a, b, c, or d). Levels range from 5 (first speech) to 12 (the most complex level Lectica measures). Phase scores are named as follows: 09d, 10a, 10b, 10c, 10d, 11a, etc. Levels 10 through 12 are considered to be “adult levels,” but the earliest phase of level 10 is often observed in middle school students, and the average high school student performs in the 10b to10c range.

In the following analysis, I’ll be identifying the highest-phase Lectical Dictionary terms in the President’s statements, showing each item’s phase. Where possible, I’ll also be looking at the form of thinking—black-and-white, if-then logic (10a–10d) versus shades-of-gray, nuanced logic (11a–11d)—these terms are embedded in.

The President’s statements

The first two statements are tweets made on 01–05–2018.

“…throughout my life, my two greatest assets have been mental stability and being, like, really smart.

The two most complex ideas in this statement are the notion of having personal assets (10c), and the notion of mental stability (10b).

“I went from VERY successful businessman, to top T.V. Star…to President of the United States (on my first try). I think that would qualify as not smart, but genius…and a very stable genius at that!”

This statement presents an argument for the President’s belief that he is not only smart, but a stable genius (10b-10c). The evidence offered consists of a list of accomplishments—being a successful (09c) businessman, being a top star, and being elected (09b) president. (Stable genius is not in the Lectical Dictionary, but it is a reference back to the previous notion of mental stability, which is in the dictionary at 10b.)

The kind of thinking demonstrated in this argument is simple if-then linear logic. “If I did these things, then I must be a stable genius.”

Later, at Camp David, when asked about these Tweeted comments, President Trump explained further…

“I had a situation where I was a very excellent student, came out, made billions and billions of dollars, became one of the top business people, went to television and for 10 years was a tremendous success, which you’ve probably heard.”

This argument provides more detail about the President’s accomplishments—being an excellent (08a) student, making billions and billions of dollars, becoming a top business person, and being a tremendous success (10b) in television. Here the president demonstrates the same if-then linear logic observed in the second tweet, above.

Summing up

The President has spoken about his intelligence on numerous occasions. Across all of the instances I’ve identified, he makes a strong connection between intelligence and concrete accomplishments — most often wealth, fame, or performance (for example in school or in negotiations). I could not find a single instance in which he attributed any part of these accomplishments to external or mitigating factors — for example, luck, being born into a wealthy family, having access to expert advice, or good employees. (I’d be very interested in seeing any examples readers can send my way!)

President Trump’s statements represent the same kind of logic and meaning-making my colleagues and I observed in the interview responses analysed for the National Leaders’ series. President Trump’s logic in these statements has a simple, if-then structure and the most complex ideas he expresses are in the 10b to10c range. As yet, I have seen no evidence of reasoning above this range.

The average score of a US adult is in the 10c–10d range.

 

Please follow and like us: