CLAS

For decades, test developers have attempted to create a generalized electronic scoring system for texts. AI-based solutions have made some inroads, but they require huge amounts of text to produce results and aren't yet reliable enough for use in individual assessment. Moreover, like the electronic scoring systems now being produced by educational test developers, they measure easily quantified aspects of texts (like vocabulary and punctuation) rather than what we really want to know—whether or not we are providing optimal support for the development of the agile optimally developed minds that are required to thrive in our rapidly changing, crisis-ridden world.

CLAS is the only computerized developmental scoring system that measures mental development reliably enough to justify its use in individual assessment contexts. It's a truly general system that can readily be adapted for use in diverse knowledge domains. Rather than measuring whatever aspects of texts are most easily quantified, CLAS measures growth along a well-validated developmental scale called the Lectical Scale. Its algorithms are based on a deep study of learning and development involving human expertise and analytics. When CLAS produces a score, it tells us where an assessment performance lands on the Lectical Scale.

As of this writing, CLAS algorithms are based on human and computer analysis of more than 50,000 assessments (and growing). At its core is the Lectical^™ Dictionary—the world's first developmental dictionary—a constantly evolving curated taxonomy of the development of meanings. (More about the Dictionary below.)

Like the LAS (Lectical Assessment System), our human scoring system), CLAS can be used:

• to score Lectical Assessments,
• to evaluate the effectiveness of interventions designed to promote cognitive development,
• to track development over time,
• to trace conceptual development,
• in comparative research on conceptual development, and
• in research on developmental processes.

CLAS has several advantages over the LAS:

1. It makes it possible, for the first time, to conduct large-scale developmental assessment,
2. It is easily expandable into new knowledge domains,
3. It is more accurate than human scorers,
4. It is entirely objective, and
5. It dramatically reduces the cost of scoring large numbers of assessments.

Since its initial introduction in 2014, CLAS has been used to score thousands of assessments for several research and evaluation projects.It now does alll of Lectica's scoring (with human oversight).

The Lectical Dictionary

The Lectical Dictionary is at the core of everything we do here at Lectica. It not only makes CLAS possible, but also helps us describe learning sequences and develop learning resources like skill maps and formative report feedback.

The Lectical Dictionary is composed of units of meaning called "Lectical Items"—words or phrases like "evidence" and "reliable evidence" that carry meaning. Each Lectical Item is assigned to a Lectical Phase (1/4 of a Lectical Level), based on a combination of empirical evidence, the judgment of our analysts, and a variety of helper algorithms. The goal is to assign items to the lowest level at which the simplest meaning they carry is likely to be useful.

For example, an examination of items containing the word evidence reveals easily observed progressions in the development of its meaning, such as the following:

• Phase 09b: something that I know is true
• Phase 09c: good information that comes from something people have seen or proved (same as a fact)
• Phase 09d: more or less proven facts that you can use to persuade others
• Phase 10a: information that comes from good research and can be used to support arguments
• Phase 10b: information that comes from different kinds of research and sources and needs to be evaluated before you use it to support arguments

Lectical Dictionary entries begin with first speech and cover the full span of development. In addition to being assigned to a Lectical Phase, many of the items in the Lectical Dictionary have been assigned to thematic strands such as deliberation, conflict resolution, the physics of energy, or evidence. For example, at this writing, over 35,000 terms in the Lectical Dictionary relate to evidence. Longitudinal and cross-sectional analyses of sequences like the evidence sequence have demonstrated that each successive conception builds upon previous conceptions (Dawson & Gabrielian, 2003; Dawson- Tunik, 2004). These findings are entirely consistent with the developmental theory upon which the dictionary is based (Fischer, 1980; Piaget, 1985), and suggest that Lectical Items assigned to a particular phase can be said not only to represent the understandings of that phase but also play the role of building blocks for future conceptions. Because of this, when we look at the distribution of Lectical Items across phases within a given performance, we are to some extent, looking at the historical pattern of an individual’s development.

The rate of Dictionary development has increased as what we learn about patterns in the acquisition of concepts is gradually integrated into our methods and technology. For example, we have learned that there are regularities in the progression of verb conjugation within particular developmental levels and that, in some cases, new single-word Lectical Items typically don’t appear next to some conjunctions until the phase following the phase to which they have been assigned. When adequately regular, patterns like these allow a degree of automation in the curation process. (We call this process lexicating™.) The Lectical Dictionary is continuously monitored, refined, and added to by trained analysts. As we build new assessments in new subject areas, the Lectical Dictionary is becoming an increasingly comprehensive curated taxonomy of meanings.

Unlike the purely computational scoring algorithms produced through the analysis of "big data" analysis or "meta-analytics," CLAS is not a form of AI. Even now, its ongoing maintenance and development requires a great deal of expert human judgment. In fact, humans make all judgments about the placement of Lectical Items, which requires the analysis of meaning (something AI can't do). 

If you'd like to know more about CLAS, please contact us.