Innovation in Language Learning

This study compares the scoring consistency and feedback quality of two generative AI tools (DeepSeek and Doubao) to determine whether it is feasible to integrate them with human evaluation in academic essay assessment. A mixed-methods approach was used to analyze 100 IELTS Task 2 essays across CEFR proficiency levels from A2 to C1, combining quantitative scoring with qualitative feedback analysis. Results from repeated measures ANOVA and Friedman’s test showed significant scoring differences. Human raters awarded higher scores than Doubao, whereas DeepSeek scores did not significantly differ from human evaluations. For feedback quality, AI tools (especially DeepSeek) outperformed humans in terms of clarity and constructiveness, with raters awarding maximum scores exclusively to AI feedback. However, human feedback excelled in specificity for surface-level errors and contextualized analysis of higher-order concerns such as evidence integration. Thematic analysis demonstrated AI's competence in action-oriented tone and structural recommendations, while humans offered insightful criticisms of the argument's relevance and evidence use. These findings support a hybrid model where AI handles formative feedback on structure and mechanics, and humans concentrate on summative evaluation of critical thinking and disciplinary nuances, enhancing efficiency while preserving educational rigor.