Text complexity in curricula and language assessments

Abstract

Theories of language development assume that children are exposed to increasingly complex vocabulary, morphosyntax, and discourse in educational contexts, and thus typically acquire these complex linguistic concepts and structures. These theories further inform assessments and interventions for children with language disorders, as well as general educational benchmarks.

Clinical and educational assessments increase in linguistic complexity with age due to this assumption, however, there may be mismatches in: 1) overall text complexity 2) the rate at which text complexity increases 3) which aspects of text complexity increase. If language in curricula is less complex than that of assessments, students have fewer opportunities to learn tested linguistic concepts and structures. This study aims to explore this through corpus analysis of textbooks and assessments, addressing the following research questions:

1. Is text complexity congruent across grade-school curricula and standardized language assessments?
2. How does text complexity in curricula vary across grade levels?
3. How does text complexity in curricula vary across subject area?

Elementary and middle school textbooks and clinical language assessments were obtained from an American metropolitan school district conforming to Common Core Standards. Clinical language assessments frequently used by school-based speech-language pathologists were selected for analysis as well. Stimuli corresponding to each grade were selected from the assessments.

The textbooks and assessments were digitized and analyzed using computational methods drawing from traditional communication sciences & disorders measures, traditional education measures, and less frequently used computational measures. The measures assess complexity at the word-, sentence-, and discourse-level. Measures were calculated using Python scripts and Coh-Metrix.

Preliminary analyses show that textbooks have higher syntactic complexity, lexical complexity, and lower readability. Flesch-Kincaid Grade Level is higher than the targeted grade level in textbooks, and lower in assessments. Textbooks and assessments do not differ significantly on cohesion measures. Assessments are more narrative-like compared to textbooks.

Assessments and textbooks are not linguistically congruent; reading comprehension on an assessment may be less difficult than that of curricular texts. Students’ performance on reading comprehension assessments may overestimate their ability to comprehend curricular texts, emphasizing the need for support in curricula comprehension.