For decades, test developers have attempted to create a generalized electronic scoring system for texts. AI-based solutions have made some inroads, but they require huge amounts of text to produce results and aren't yet reliable enough for use in individual assessment. Moreover, like the electronic scoring systems now being produced by educational test developers, they measure easily quantified aspects of texts (like vocabulary and punctuation) rather than what we really want to know—whether or not we are providing optimal support for the development of the agile optimally developed minds that are required to thrive in our rapidly changing, crisis-ridden world.

CLAS is the first computerized developmental scoring system that measures what educators really want to know—how well learners can put their knowledge to work in real-world contexts—reliably enough to be used ethically in individual assessment contexts. It's a truly general system that can readily be adapted for use in diverse knowledge domains. Rather than measuring whatever aspects of texts are most easily quantified, CLAS measures growth along a well-validated developmental scale called the Lectical Scale. Its algorithms are based on a deep study of learning and development involving human expertise and analytics. When CLAS produces a score, it tells us where an assessment performance lands on the Lectical Scale, what the score means in terms of the development of a test-taker's understanding of and ability to apply ideas targeted by the assessment in real-world contexts, as well as what the test taker is likely to benefit from learning next.

As of this writing, CLAS algorithms are based on human and computer analysis of more than 50,000 assessments (and growing). At its core is the Lectical^™ Dictionary—the world's first developmental dictionary—a constantly evolving curated taxonomy of the development of meanings. (More about the Dictionary below.)

Like the LAS (Lectical Assessment System), our human scoring system), CLAS can be used:

• to score Lectical Assessments,
• to evaluate the effectiveness of interventions designed to promote cognitive development,
• to track development over time,
• to trace conceptual development,
• in comparative research on conceptual development, and
• in research on developmental processes.

CLAS has several advantages over the LAS:

1. It makes it possible, for the first time, to conduct large-scale developmental assessment,
2. It is easily expandable into new knowledge domains,
3. It is entirely objective, and
4. It dramatically reduces the cost of scoring large numbers of assessments.

Since its introduction in 2014, CLAS has been used to score thousands of assessments for several research and evaluation projects.

The Lectical Dictionary

The Lectical Dictionary is at the core of everything we do here at Lectica. It not only makes CLAS possible, but also helps us describe learning sequences and develop learning resources like skill maps and formative report feedback.

The Dictionary is composed of units of meaning called "Lectical Items"—words or phrases like "evidence" and "reliable evidence" that carry meaning. Each Lectical Item is assigned to a Lectical Phase (1/4 of a Lectical Level), based on a combination of empirical evidence, the judgment of our analysts, and a variety of helper algorithms. The goal is to assign items to the lowest level at which the simplest meaning they carry is likely to be useful.

For example, an examination of items containing the word evidence reveals easily observed progressions in the development of its meaning, such as the following:

• Phase 09b: something that I know is true
• Phase 09c: good information that comes from something people have seen or proved (same as a fact)
• Phase 09d: more or less proven facts that you can use to persuade others
• Phase 10a: information that comes from good research and can be used to support arguments
• Phase 10b: information that comes from different kinds of research and sources and needs to be evaluated before you use it to support arguments

Lectical Dictionary entries begin with first speech and cover the full span of development. In addition to being assigned to a Lectical Phase, many of the items in the Lectical Dictionary have been assigned to thematic strands such as deliberation, conflict resolution, the physics of energy, or evidence. For example, at this writing, over 85,000 terms in the Lectical Dictionary relate to evidence. Longitudinal and cross-sectional analyses of sequences like the evidence sequence have demonstrated that each successive conception builds upon previous conceptions (Dawson & Gabrielian, 2003; Dawson- Tunik, 2004). These findings are consistent with the developmental theory upon which the dictionary is based (Fischer, 1980; Piaget, 1985), and suggest that Lectical Items assigned to a particular phase can be said not only to represent the understandings of that phase but also play the role of building blocks for future conceptions. Because of this, when we look at the distribution of Lectical Items across phases within a given performance, we are to some extent, looking at the historical pattern of an individual’s development.

The rate of dictionary development has increased as what we learn about patterns in the acquisition of concepts is gradually integrated into our methods and technology. For example, we have learned that there are regularities in the progression of verb conjugation within particular developmental levels and that, in some cases, new single-word Lectical Items typically don’t appear next to some conjunctions until the phase following the phase to which they have been assigned. When adequately regular, patterns like these allow a degree of automation in the curation process. (We call this process lexicating™.) The Lectical Dictionary is continuously monitored, refined, and added to by trained analysts. Over time, as we build new assessments in new subject areas, the Lectical Dictionary will become an increasingly comprehensive curated taxonomy of meanings.

Unlike the purely computational scoring algorithms of conventional "big data" analysis or "meta-analytics," CLAS's algorithms are developed through a human/machine collaboration. CLAS and our analysts are constantly engaged in "conversations" about the placement of Lectical Items. CLAS's algorithms represent our current knowledge, and our analysts interact with CLAS to integrate new knowledge. This partnership approach required 20 years of human labor and learning up front, but the resulting system produces more accurate and meaningful scores than those produced with analytics alone. Importantly, CLAS not only produces accurate scores; it also makes it possible to inquire deeply into the development of meaning and the growth of the human mind.

If you'd like to know more about CLAS or have ideas about how CLAS technology could enhance your work, please contact us.