Lectica's research methods

The best assessments are grounded in a deep understanding of how relevant concepts and skills develop over time. For example, if a curriculum is intended to develop decision-making skills, assessments should evaluate developmental progress in acquiring those skills. To do this well, it is essential to meet 3 objectives:

• documenting the range of skills required for good decision-making in various life or job roles;
• tracing the developmental pathways through which these skills optimally develop; and
• identifying effective strategies for promoting the development of these skills.

We have designed an iterative methodology that is employed to build the knowledge base required to meet these objectives. We call this methodology developmental maieutics. It involves several steps:

1. establishing a collaborative relationship with clients (researchers, end users, and content experts);
2. leveraging the expertise of clients to identify assessment goals and target constructs;
3. working closely with clients and domain experts to design a pool of open response items;
4. administering items to a diverse sample of several hundred individuals from each targeted educational level (often grades 4 – 21);
5. scoring performances with LAS or CLAS to determine where they belong on our 13 level learning scale (the Lectical Scale™);
6. lexicating™ words and phrases from the new performances that are not yet in our developmental dictionary (the Lectical Dictionary™). Lexicating, involves identifying new words and phrases and assigning each to the first developmental phase (1/4 of a level) at which they are likely to be used. We also assign new words and phrases to one or more thematic areas, eg., evidence, the physics of energy, deliberation, ethics, etc.);
7. using the results to describe detailed learning sequences for each new theme and update learning sequence descriptions for existing themes;
8. based on these learning sequences, developing complementary curricular materials and report content;
9. administering items for a second time (12 months after the first test time) to test the accuracy of learning sequences, the placement of new words and phrases in the Lectical Dictionary, and the usefulness of report feedback; and
10. iterating continuously based on user data and feedback.

With adequate funding, a fully developed set (2-4 alternate forms of the same assessment) of Lectical Assessments (a.k.a. DiscoTests), complete with curricular materials, learning resources, and reports, can be developed in about 2 years. During the development phase, new assessments are made available to the public free of charge, through the CLAS demo.

To learn more about skill levels, see the hierarchical complexity and skill level pages.