DiscoTest Initiative rationale

Rationale & evidence for our nonprofit mission

Summary

Many 21st century challenges are highly complex, contested, and time-sensitive. To address them, democratic societies need citizens whose mental models and reasoning skills make it possible for them to (1) appreciate the complexity of these challenges, (2) take into account both short and long-term implications, (3) envision tradeoffs among personal, local, and global interests, and (4) thrive in rapidly changing environments. Unfortunately, our current educational system falls short of supporting the optimal development of these capabilities [1,9] and recent efforts to improve the system have not succeeded [2]. Lectica has developed a solution that leverages a robust theroetical framework and new educational technology to foster systemic change [3]. Lectica’s lever—the DiscoTest Initiative[4]—will catalyze the widespread adoption of educational practices that provide optimal support for the development of effective and successful 21st century citizens [5].

Team purpose

The purpose of the DiscoTest Initiative is to prepare K-12 students for the 21st century by taking formative assessment [5] to scale:

  1. rewarding educational practices that reinforce students’ inborn passion for learning;
  2. equipping students with skills and dispositions for negotiating the complexities and challenges of today’s world—skills for thinking, learning, evaluating, deciding, acting, communicating, interacting, and creating;
  3. documenting the development of these skills [6]; and
  4. providing students, teachers, and parents with feedback and resources that support optimal development.

We do this by building and delivering online learning tools called DiscoTests, each of which gently nudges students and teachers toward more optimal ways of learning and teaching.

DiscoTests will address many of the “big ideas” captured in state and national standards, including math, the sciences, social studies, literature, and the arts.

Problem & solution

Problem

Our current educational system, as a whole, falls short of supporting the optimal development of essential 21st century capabilities [1].

  • Most students spend too much of their time learning facts, procedures, rules, and vocabulary, and too little of their time building the kind of knowledge required for thinking, learning, evaluating, deciding, acting, communicating, interacting, and creating [2].
  • Today’s high stakes standardized tests, which were developed to improve public education by holding educators accountable, are more right answer focused than skill focused. They tend to encourage teaching that prepares students to provide enough right answers [2].
  • Too great a focus on right answers produces shallow learning—learning that doesn’t go deep enough to become “sticky” or useable [7].
  • There is no parallel incentive—no fully scalable technology—that systematically encourages students to learn in ways that support deep, highly networked, embodied [8], useable knowledge—what we call robust learning [9].
  • Robust learning not only results in useable knowledge, but also provides the foundation for a lifetime of learning and development—the kind of learning that allows people to grow into complex thinkers equipped to grapple effectively with 21st century [7,9] challenges—complex issues that lack simple “right answers”.
  • Lectica plans to transform educational practice by rewarding robust learning—at scale. Just as conventional high-stakes testing has produced an increased focus on pedagogy that rewards shallow learning, the right kind of learning tools can catalyze the widespread adoption of educational practices that support robust learning [2,3]. References
Solution

Our solution is an innovative technology that builds unprecedented knowledge about the development of skills and knowledge while delivering powerful learning tools called DiscoTests.

  • DiscoTests help to ensure that what a student learns today is learned deeply and richly enough to be useable and enduring. This kind of learning, which we call robust learning, also provides a foundation for future learning that, over time:
    • supports increasingly complex, sophisticated ways of understanding our world; and
    • builds skills for putting those ways of understanding to work [4,7,9,10].
    • DiscoTests are richly diagnostic, providing information about students’ learning edges—what they’re poised to benefit most from learning next [11].
  • They are radically formative [5]:
    • They provide immediate feedback, triggering the dopamine-opioid cycle that reinforces our inborn motivation to learn [5,11,12].
    • They optimize learning by supporting Virtuous Cycles of Learning—cycles of goal setting, information gathering, application, and evaluation [11,13].
  • They are all calibrated to a single theoretically derived learning metric—making it possible to track growth on a non-arbitrary scale [14,24]. All students, regardless of their rates of growth, will be able to see themselves as learners [5,6,14].
  • Each DiscoTest can be used in multiple grades, often grades 4–12.
  • The results of research we conduct to build each DiscoTest is stored in a knowledge base (developmental dictionary) we call the Lectical Dictionary [15]. It is used to trace learning trajectories, build learning resources, and score assessments.
  • All DiscoTests are free to individual K-12 educators everywhere.

Impact

  1. Testing drives instruction. Tests that measure robust learning reward teachers for fostering robust learning [2,5].
  2. We have authored peer-reviewed articles about several validation studies [16].
  3. Higher scores on our learning scale predict greater literacy [21].
  4. Virtuous Cycles of Learning (VCoLs) support robust learning [7]. There is growing evidence that the more students use VCoL, the more effectively they grow.
  • On average, high school seniors in programs that do the best job supporting robust learning are about 3 years ahead (on our learning scale [14]) of seniors in schools that do an average job—after taking SES into account [9].
  • Even in adult contexts, educational programs that incorporate more VCoLs produce more learning than programs incorporating fewer VCoLs [16].
  1. Robust learning matters.
  • Students in programs that do the best job supporting robust learning are more likely to continue developing well into adulthood [9].
  • Adults who score higher on our learning scale are more likely to be promoted into higher paying or more sophisticated jobs [18].
  1. The DiscoTest Initiative is about much more than testing—it’s about a new way of understanding and supporting learning. What we are learning has:
  • allowed us to build highly effective learning tools;
  • led us to the mighty micro-VCoL;
  • made it possible to create the world’s first developmental dictionary and electronic developmental scoring system [15];
  • provided us with an increasingly sophisticated understanding of the shape of development;
  • allowed us to rapidly document, in detail, how students build specific meanings over time; and
  • has even allowed us to create an improved spell checker.

We’re confident that there is more to come.

References

[1]

Madsbjerg, C., Rasmussen, M. B. (2014) An Anthropologist Walks into a Bar… Harvard Business Review, March.

Dawson, T. L., & Stein, Z. (2004). National Leadership Study results. Hatfield, MA: Developmental Testing Service, Inc.

Dawson, T. L. (2009). Task demands and capabilities (the complexity gap), Lectica FAQ.

Hutchins, Giles, Phoenix Rising… The time has come. The Nature of Business.

[2]

FairTest; Mulholland, Quinn (2015). The case against standardized testing. Harvard Political Review, May 14.

[3]

For examples, see the Donella Meadows Institute and Archive, her important article, Leverage points: places to intervene in a system, and the numerous publications on this issue that have emerged out of MIT.

[4]

Dawson, T L., Stein, Zachary (2011). Virtuous cycles of learning: Redesigning testing during the digital revolution. Originally presented at the Ettore Majorana Center for Scientific Culture, Erice (Sicily), Italy, International School on Mind, Brain, and Education.

[5]

NCTE Executive Committee (2013). NCTE position statement: Formative assessment that truly informs instruction, October 13.

[6]

Dawson, T. L. (2016). A new kind of report card.

[7]

Schwartz, M. (2014) Khan academy: The illusion of understanding: Journal of Asynchronous Learning Network. 17, 4, 67-80.

Schwartz, M. S., Sadler, P. M., Sonnert, G. & Tai, R. H. (2009). Depth versus breadth: How content coverage in high school science courses relates to later success in college science coursework. Science Education, 93, 5, 798-826.

[8]

Dawson, T. L. (2016). What do we mean by embodied learning?

Kontra, C., Goldin-Meadow, S., & Beilock, S. L. (2012). Embodied learning across the lifespan. Topics in Cognitive Science, 4(4), 731–739.

[9]

Dawson, T. L. (2016). Are our children learning robustly?

[10]

Stein, Z., Dawson, T. L., & Fischer, K. W. (2010). Redesigning testing: Operationalizing the new science of learning. In New science of learning: Computers, cognition and collaboration in education.

Dawson, T. L., & Fischer, K. W. (2006). Implications of assessment for learners. Measurement, 4, 4.

[11]

Dawson, T. L. (2016) Educational assessment: A cognitive science approach. Presented to the Mind, Brain, and Education program at the University of Texas at Arlington, April.

[12]

Dawson, T. L. (2016). 500 Addicted babies

Berridge, K. C. and Robinson, T. E. (1998). What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Research Reviews, 28, 309–369.

[13]

Dawson, T. L. (2016). VCoL+7

Learning cycles, Wikipedia
[14]

Dawson, T. L. (2016). About Lectical Levels

Dawson-Tunik, T. L. (2004). A good education is: The development of evaluative thought across the life-span. Genetic, Social, and General Psychology Monographs, 130, 4-112.

Dawson, T. L. (2004). Assessing intellectual development: Three approaches, one sequence. Journal of Adult Development, 11, 71-85.

Dawson-Tunik, T. L., Commons, M., Wilson, M., & Fischer, K. (2005). The shape of development. The European Journal of Developmental Psychology, 2, 163-196.

Dawson, T. L., & Stein, Z. (2008). Cycles of research and application in education: Learning pathways for energy concepts. Mind, Brain, & Education, 2(2), 90-103.
[15]

Each term or phrase in the Lectical Dictionary is—always provisionally—assigned to one of 28 phases on our learning metric. Each DiscoTest that’s taken expands and refines this dictionary. In essence, we’re building a sophisticated developmental taxonomy of meanings. This Lectical Dictionary™ has already taught us a great deal about learning. In fact, it began its life as a way of studying patterns we observed in student performances.

The fact that the Lectical Dictionary is human vetted makes it stand out from other attempts to look at language developmentally at scale. Lexiles®, for example, were created purely computationally. Our iterative systems-informed approach integrates technology, developmental theory, and human expertise. We believe this is why we have achieved a practical and fully scalable solution to electronic essay scoring ahead of our much better funded “competitors.”

[16]

Articles, Validity and reliability (Lectica website)

[17]

Based on formal and informal polling of teachers, including our Teacher Advisory Board, we believe that teachers are likely to adopt new tools if they:

  1. help solve a problem teachers are already aware of;
  2. resonate with their explicit and tacit knowledge of learning;
  3. can be fit into the standards relevant to their context;
  4. free up time, so they can have more direct contact with students; and
  5. make a real, observable difference.

With these factors in mind, we have asked teachers what DiscoTest features are most important. Their answers: (a) there should be several available in each subject area; (b) they should focus on “core concepts”, particularly those that are difficult to teach; (c) they should be easier to use than their existing assessments; and (d) they should be associated with truly useful learning resources.

[18]

Dawson, T. L (2016). Leader decisions series

[19]

Texas rejects critical thinking, Oklahoma rejects AP History

[20]

Fuhs, C. J. (2015). A latent growth analysis of hierarchical complexity and perspectival skills in adulthood.Santa Barbara, CA, Fielding Graduate University.

Heikkinen, K. M. (2014). The Development of Social Perspective Coordination Skills in Grades 3-12. Graduate School of Education. Cambridge, MA, Harvard. Ph.D.: 208.

Stein, Z. (2014). Tipping the scales: Social justice and educational measurement. Harvard Graduate School of Education. Cambridge, MA, Harvard. Ed.D.: 288.

Van Rossum, Z. (2013). The development of social perspective taking and leadership decision-making in city government managers. Teachers College. New York, Columbia University. Ed.D.: 285.

[21]

Contributions of academic language, perspective taking, and complex reasoning to deep reading comprehension

Selected funders

IES (US Department of Education)

The Spencer Foundation

NIH

Dr. Sharon Solloway

The Simpson Foundation

The Leopold Foundation

Donor list

Selected clients

Glastonbury School District, CT

The Ross School

Rainbow Community School

The Study School

Long Trail School

The US Naval Academy

The City of Edmonton, Alberta

The US Federal Government

Advisory Board

Antonio Battro, MD, Ph.D., One Laptop Per Child

Marc Schwartz, Ph.D. and former high school teacher, University of Texas at Arlington

Mary Helen Immordino-Yang, Ed.D., University of Southern California

Willis Overton, Ph.D., Temple University, Emeritus