Automated Integrative Complexity: Current Challenges and Future Directions

Automating integrative complexity is fraught with many challenges. To address these challenges, we discuss the tension between a specificity approach and a more flexible multiple-pass approach, the multifaceted nature of the complexity construct, the gold standard for complexity measurement, the difficulty of human scoring and its consequences for automation, and some ways forward for creating the best complexity measurements. In so doing, we present new data demonstrating (1) initial evidence for the validity of a new automated system for measuring two different forms of complexity (elaborative and dialectical), (2) the danger of constructing measurements in a purely ad hoc fashion that ignores prospective testing, (3) human-to-computer correspondence is in part a function of human-to-human correspondence, (4) human-to-computer correspondence increases systematically as one uses tests with larger units of analysis, and (5) the lack of correspondence of different systems (both human and automated) may occur in part because they were designed for different units of analysis.