Skip to main content

Cornell University

Office of the Dean of Faculty

Connecting & Empowering Faculty

AI + grading policy

.

Draft proposal

AI + grading: proposed Cornell-wide ‘minimum floor’ policy

Grading is an essential part of teaching practice at Cornell and implicates several of the Cornell GenAI principles in teaching and learning, including the integrity of the faculty-student relation; and the centrality of faculty judgement and expertise in the classroom.

Research into the ability of GenAI tools to grade student work show significant limitations, including inconsistent reliability, systemic bias patterns, and difficulty evaluating nuanced content. (See appendix for details). Given this, GenAI should not be used as the sole evaluator for summative assessment, and faculty should carefully consider whether AI-assisted grading is appropriate given these limitations.

When GenAI is used to assist with grading, instructors should review all AI-generated data with particular attention to content quality, argumentation, and original thinking. This is in addition to other critical considerations, including students’ right to privacy (do not use an unprotected system), the need to disclose to students any AI-assisted grading, and students’ right to appeal.

With this in mind, any use of AI in grading at Cornell must meet the following minimum requirements:

  • ‘Faculty-in-the-loop’ (an responsibility requirement): instructors retain responsibility for the soundness and accuracy of all grades used to calculate course credit. Where AI tools are used to assist grading, members of the teaching team retain an active role in reviewing and evaluating student work.
  • Transparency (a disclosure requirement): any use of AI in grading must be clearly disclosed to students and to leadership of the relevant department (e.g. Chair or Director). Any TA use of AI in grading must be reviewed and approved by the course instructor.
  • Right of (human) appeal (an appeal requirement): in cases of dispute, students retain the right of appeal and reassessment from members of the course teaching staff (faculty, TAs, etc.).
  • Grade and student privacy (a security requirement): all faculty are responsible for maintaining security of grades and private student information, including responsibilities under FERPA, etc. Faculty are strongly encouraged to use Cornell-approved tools and instances (Microsoft CoPilot; approved GenAI pilot tools; etc.). No use of AI in grading is permitted where the security and integrity of student information cannot be verified and maintained.

Beyond meeting this minimum floor, individual faculty, departments and colleges may choose to adopt additional policies (including restrictions) around the use of AI in grading that are specific to course, disciplinary, department or college needs.  These should continue to meet the standards of responsibility; disclosure; appeal; and security articulated above.  Collective policies (if any) at the department or college level should be clearly communicated to faculty, students, and the relevant college leadership.

Elements that faculty or departments might consider in thinking further around appropriate or inappropriate uses of AI in grading include:

  • distinctions between formative assessment (e.g. feedback) and summative assessment (e.g. a grade);
  • distinctions of scale (low-stakes vs. high-stakes assignments);
  • distinctions by class size or scale;
  • distinctions by student level (freshman to doctoral);
  • distinctions by modality or type (text, math, code, video/audio, etc.).
  • distinctions between augmenting vs. replacing instructor feedback and judgement
  • variabilities in tool performance and appropriateness

Appendix

Research Review (January 2026)

LIMITATIONS of AI

Agreement with Human Graders. Studies have found weak correlations between AI and human scoring (Bui & Barrot, 2025). Interestingly, even when AI scores appear similar to human grades, deeper analysis reveals significant differences in score distribution and evaluation patterns (Flodén, 2025). LLM’s assessment of “substance and structure” in written work is where they differ most from human evaluators (Yavuz et al., 2025).

Test-Retest Reliability. ChatGPT produced questionable test-retest reliability when scoring the same essays twice with a four-week interval compared to human consistency (Bui & Barrot, 2025). When essays were graded repeatedly by ChatGPT (including GPT4), a “substantial” portion received different grades across multiple grading sessions (Flodén, 2025, Pack, 2025) perhaps due to updates made to the proprietary model used.

Scoring Biases. AI exhibits “proportional bias” ie: being too lenient with weak work but too strict with high-quality work (Wetzler et al., 2025). Even fine-tuned models showed leniency bias, being more generous with good and average essays than human raters (Yavuz et al., 2025). Premium models tend toward higher median grades overall compared to human graders (Mendonça et al., 2025). In one study, ChatGPT failed to differentiate essays by proficiency level (Bui & Barrot, 2025).

Context Recognition Failures. LLMs miss context. Floden (2025) finds they often award high scores to answers that are factually correct but don’t actually address the question asked. AI particularly struggled with questions closely related to course lectures but performed better on more general questions (ibid).

Model Drift and Instability. As noted above, updates to models over time affect consistency and accuracy in unpredictable ways (Bui & Barrot, 2025). Models can be reliable over time or change performance between grading sessions, meaning scores may not be stable across grading periods (Pack et al., 2024).

CONDITIONS FOR POSITIVE RESULTS

Model Selection. Premium models like GPT-4 significantly outperform both older free versions like GPT-3.5 and open-source alternatives (Mendonça et al., 2025; Wetzler et al., 2025; Pack et al., 2024; Tang et al., 2024). Only premium models achieve practical equivalence to human scoring within acceptable thresholds (Mendonça et al., 2025). GPT-4 showed the highest correlations with human raters and excellent interrater reliability across two time points approximately one month apart (Pack et al., 2024). However, premium models also tended toward higher median grades overall compared to humans (Mendonça et al., 2025). This may change when studies of newer models are reviewed and published.

Prompt Engineering. Prompt design affects performance with the most sophisticated strategy shown as combining rubric criteria, detailed rationale requirements, and three annotated example essays. This process improved GPT-4’s accuracy compared to basic prompts (Tang et al., 2024). Few-shot prompting with sample essays improved performance compared to zero-shot approaches (Tang et al., 2024; Flodén, 2025; Mendonça et al., 2025). Lower temperature settings produced more reliable scores aligned with human evaluations, while higher temperatures introduced variability (Tang et al., 2024). Fine-tuned ChatGPT (a more detailed prompt and lowering the temperature to enable a more focused and deterministic output leading to greater consistency in scoring) achieved high inter-rater reliability compared to default ChatGPT and Google’s Bard when grading EFL essays (Yavuz et al., 2025). Tang et al (2024) outline several functional prompting techniques.

Rubrics and Context. As with human-only grading, studies of LLMs and assessment show positive results when comprehensive analytical rubrics (with specific criteria for each dimension being assessed) are used (Pack et al., 2024; Yavuz et al., 2025). Providing course materials and context through Retrieval-Augmented Generation (RAG) improved performance for text-based questions (Díaz et al., 2025; Mendonça et al., 2025).

PERFORMANCE BY CATEGORY

Positive Performance: Technical Aspects of Writing, Ideas and Organization. GPT-4 performed best on dimensions assessing overall ideas and organizational structure, approaching or slightly exceeding human inter-rater reliability for these dimensions. LLMs can recognize logical flow and main concepts reasonably well under controlled conditions (Tang et al., 2024). LLMs were found by Yavuz (2025) to be proficient at assessing technical aspects of writing such as punctuation, capitalization, and spelling, in a manner consistent with human experts.

Subject Matter Differences. LLMs showed more promise for structured, objective evaluations like programming code compared to subjective assessments requiring creativity and complex reasoning (Mendonça et al., 2025). Premium LLMs showed closest alignment with human grading on objective, code-based evaluations (Mendonça et al., 2025). AI struggles with essays requiring appreciation of creativity, individual differences, and complex psychological reasoning (Wetzler et al., 2025). AI may favor verbose answers with little substance while undervaluing concise, substantive responses (Flodén, 2025).

STUDENT TRUST in AI GRADING

Perceptions and Trust. In one study, students struggled to distinguish AI from human grading and their perceptions of quality were influenced by what they believed to be the source (Thomas et al., 2026). When students incorrectly believed human grading was AI-generated, their confidence dropped suggesting “reverse automation bias” (Thomas et al., 2026). Thomas also found that exposure to AI grading actually reduced students’ confidence in AI scoring, with perceptions becoming more critical after disclosure (Thomas et al., 2026). Students familiar with AI expressed more skepticism, suggesting that familiarity reveals limitations rather than building confidence (Thomas et al., 2026).

REFERENCES

Bui, Ngoc My, and Jessie S. Barrot. 2025. “ChatGPT as an Automated Essay Scoring Tool in the Writing Classrooms: How It Compares with Human Scoring.” Education and Information Technologies 30: 2041–2058. https://doi.org/10.1007/s10639-024-12891-w

Díaz, Brayan, Gongfan Chen, Edward Jaselskis, and Cesar Delgado. 2025. “Supporting Generative AI Literacy: Exploring the Pedagogical Roles Students Assign ChatGPT and Impact on Course Grades.” Comunicar 82 (33): 46–61. https://doi.org/10.5281/zenodo.15993999

Flodén, Jonas. 2025. “Grading Exams Using Large Language Models: A Comparison Between Human and AI Grading of Exams in Higher Education Using ChatGPT.” British Educational Research Journal 51 (1): 201–224. https://doi.org/10.1002/berj.4069

Mendonça, Pedro C., Filipe Quintal, and Fábio Mendonça. 2025. “Evaluating LLMs for Automated Scoring in Formative Assessments.” Applied Sciences 15 (5): 2787. https://doi.org/10.3390/app15052787

Pack, Austin, Alex Barrett, and Juan Escalante. 2024. “Large Language Models and Automated Essay Scoring of English Language Learner Writing: Insights into Validity and Reliability.” Computers and Education: Artificial Intelligence 6: 100234. https://doi.org/10.1016/j.caeai.2024.100234

Tang, Xiaoyi, Hongwei Chen, Daoyu Lin, and Kexin Li. 2024. “Harnessing LLMs for Multi-Dimensional Writing Assessment: Reliability and Alignment with Human Judgments.” Heliyon 10 (14): e34262. https://doi.org/10.1016/j.heliyon.2024.e34262

Thomas, Mackenzie L., Seyma N. Yildirim-Erbasli, and Shruthi Hariharan. 2026. “Exploring Undergraduate Students’ Perceptions of AI vs. Human Scoring and Feedback.” The Internet and Higher Education 68: 101052. https://doi.org/10.1016/j.iheduc.2025.101052

Wetzler, Elizabeth L., Kenneth S. Cassidy, Margaret J. Jones, Chelsea R. Frazier, Nickalous A. Korbut, Chelsea M. Sims, Shari S. Bowen, and Michael Wood. 2025. “Grading the Graders: Comparing Generative AI and Human Assessment in Essay Evaluation.” Teaching of Psychology 52 (3): 298–304. https://doi.org/10.1177/00986283241282696

Yavuz, Fatih, Özgür Çelik, and Gamze Yavaş Çelik. 2025. “Utilizing Large Language Models for EFL Essay Grading: An Examination of Reliability and Validity in Rubric-Based Assessments.” British Journal of Educational Technology 56 (1): 150–166. https://doi.org/10.1111/bjet.13494

PROCESS

Google search, Cornell library database, and Claude Sonnet 4.5 were used to compile a spreadsheet selection of literature focused on Automated Essay Scoring (AES) and LLMS in higher ed. Peer-reviewed articles (no pre-prints) showing limitations/concerns, positive results, or both were selected. These studies surfaced a range of methodologies and models, (closed source, open source) techniques, (zero shot, few shot, detailed prompting) and contexts. Selected papers were published within the past 2 years. After narrowing references by reading abstracts and occasionally, looking more closely into methods, Paperpile (licensed and not AI) was used to organize and quickly annotate documents to surface key topics. Annotations were exported, and Claude Sonnet 4.5 Pro helped to organize these into categories. Claude Sonnet 4.5 was used iteratively to co-author a long version of this doc and then again to condense. Language was edited and the synthesis was verified against original sources. Claude Sonnet 4.5 cleaned the citations list which was compiled from a Paperpile export. The final list of references was then verified.

 

EPC review

Educational Policy Committee (EPC)

Response to AI + grading: proposed Cornell-wide ‘minimum floor’ policy

17 September 2026

The committee appreciates the considerable effort that went into developing this proposal and
agrees that Cornell should establish a minimum institutional framework governing the use of AI
in grading. We support the policy’s emphasis on faculty responsibility and transparency, student
appeal, and the protection of student privacy.
At the same time, our discussion highlighted several broader concerns that sit at the core of our value proposition as educators.
Reinforcing the educational purpose of assessment We encourage strengthening the preamble to emphasize that the value of higher education lies not only in teaching students to produce answers, but also in cultivating their capacity for informed judgment. Assessment is more than a mechanism for assigning grades; it is a core part of teaching and learning through which faculty evaluate students’ reasoning, use of evidence, judgment, and development within and beyond their disciplines. Because these outcomes are central to our mission, assessment must continue to rely on the expertise and judgment of its educators.
Faculty judgment should remain central even as technology evolves We believe that the justification for human involvement in grading should extend beyond the current limitations of AI systems. While these concerns are important, several committee members noted that the rationale for faculty involvement should not depend solely on current technological capabilities and constraints. Even if future systems become more accurate and
reliable, Cornell would still need to address the role of disciplinary expertise and human
judgment in assessment. Faculty are not simply responsible for verifying whether answers are
correct. They evaluate qualities such as intellectual growth, professional judgment, creativity, ethical reasoning, interpretation of evidence, and decision-making under uncertainty. These forms of evaluation are central to the educational mission of the university and should remain central to policies governing AI-assisted grading.
“Faculty-in-the-loop” requires further definition
The committee believes the policy would benefit from further clarification of what constitutes
meaningful “faculty-in-the-loop” oversight. Retaining formal responsibility for grades is
important, but meaningful oversight requires more than simply approving AI-generated
evaluations. The policy should articulate how and why faculty expertise and judgment remain
actively engaged in the assessment process.
Distinction beyond disciplines, driven by the nature of the assessment The committee encourages greater attention to differences among assessment types. While disciplinary differences certainly exist, an equally if not more important distinction is the nature of what is being assessed. AI is well suited to objective, highly structured tasks, such as programming exercises or computational problems where there is a “correct” answer. Greater caution is warranted when assessment requires interpretation, creativity, judgment, ethical
reasoning, contextual understanding, or evaluation of complex and ambiguous intellectual processes. We think that the policy should acknowledge that there are contexts in which AI assisted grading may be entirely appropriate, while in others it may be inappropriate regardless of technological capability. The policy should explicitly acknowledge that use of AI is optional. We are concerned that faculty may feel increasing pressure to adopt AI tools. We recommend
making explicit that this policy establishes conditions under which AI use may be permissible, not an expectation that faculty should use AI in grading. Faculty should remain free to determine that AI has no appropriate role in particular courses, assignments, or disciplines.
Formative versus summative assessment and class size should not drive distinctions
The draft distinguishes possible use cases for formative and summative assessment. We agree that consequences for students differ between these categories and that summative evaluation warrants particular care. That said, this distinction emphasizes grades rather than learning. Formative feedback can have profound effects on student learning trajectories, confidence, revision practices, and intellectual development. As such, the need for human oversight should depend not only on whether an assessment is formative or summative, but also on the degree of evaluative judgment involved. Feedback that shapes learning may warrant meaningful human involvement regardless of whether it directly contributes to a final grade.
Similarly, while practical implementation challenges differ between small and large courses, several members expressed concern that class size is fundamentally a resource consideration rather than an educational principle. Students in large courses should not receive a fundamentally different standard of evaluation than students in small courses simply because of scale. If meaningful human evaluation is necessary to assess student work, that requirement should hold regardless of course size. Course scale may appropriately influence implementation strategies, but not the educational principles underlying assessment.
Security and privacy remain a concern
The Committee appreciates the acknowledgement that any AI system used must adhere to FERPA standards, and that a Cornell Sandbox may meet these. However, given recent data breaches across LLM platforms, some members are concerned that existing safeguards regarding privacy and security are not sufficient. Some faculty may conclude that the ethical risks associated with exposing student work to AI systems outweigh the potential benefits. The policy should acknowledge the rapidly changing nature of the technology Committee members emphasized that both AI capabilities and institutional understanding of AI and its uses continue to evolve rapidly. The policy should recognize that it itself is a framework for decision-making and not a final resolution of ongoing questions about teaching, learning, and assessment. Flexibility and periodic review will likely be necessary as technologies, evidence, disciplinary practices, and educational goals continue to develop.
Conclusion The committee supports the goal of establishing a university-wide minimum framework governing the use of AI in grading. We particularly support the principles of responsibility, transparency, student appeal, and protection of student privacy. At the same time, we encourage strengthening the policy’s educational foundations by placing greater emphasis on the purposes of assessment, the centrality of faculty expertise and judgment, disciplinary and assessment-specific differences, and the continuing importance of the faculty-student relationship. We believe these considerations will make the policy more adaptable to coming changes, more broadly applicable across disciplines, and better aligned with Cornell’s educational mission.

Use of AI in Grading and Assessment at Cornell: A Proposed University-Wide Minimum Floor Policy (updated November 2026)

About this document:
This is the most recent version (Nov 2026) of AI + Grading, minimum fl oor policy. Initial versions of this document were developed by faculty, students and staff on the AI and Education Working Group in Fall 2025, and reviewed by university leadership (Provost Staff and Deans Council), the Faculty Senate Educational Policy Committee and the Student Assembly in spring, summer and fall 2026. The appendix includes an updated review of research relating to AI, grading and student assessment that may be useful to university and faculty deliberations in this space.
AI + Grading: A Proposed Minimum Floor Policy
Purpose of grading and assessment
Assessment and grading of student work is central to teaching and learning at Cornell. Faculty are responsible not only for examining student mastery of learning objectives, but for evaluating their students intellectual growth and professional judgment, their creativity and ethical reasoning, and their interpretation of evidence and decision-making under uncertainty. Per Cornell’s AI and Education Core Principles, including ‘the centrality of faculty judgement and expertise in the classroom’ and ‘the integrity of the faculty-student relation’, instructors who look to engage AI technologies in their evaluation of student work are expected to continue to rely on and center their disciplinary expertise and judgement while retaining full responsibility for and substantive engagement in the assessment process – a ‘faculty-in-the-lead’, not a ‘faculty-in-the-loop’ model. Assessment and feedback on work is one of the main ways faculty and students engage with each other’s thinking, and any use of AI in grading or feedback should preserve that relationship: students should be able to trust that their work has been read and judged by the people responsible for teaching them.
Scope of policy:
This document establishes a minimum institutional framework describing the conditions under which AI use in grading and feedback may be permissible at Cornell. Pending review and approval by the Cornell University Faculty Senate and the General Faculty Council and Executive Faculty Council at Weill Cornell Medicine, it is intended to apply to all educational units across Cornell’s operations in Ithaca, New York, and elsewhere. It creates no expectation that faculty or members of the teaching team use AI in grading, and faculty remain free to determine that AI has no appropriate role in particular courses, assignments, or disciplines. They need offer no justification for that decision.
This policy applies to both summative assessment (work that contributes to a grade) and to formative feedback (comments that shape learning, whether or not they contribute to a grade). While they are not a final resolution of ongoing questions about the use of AI in teaching, learning, and assessment, they establish a clear minimum floor that any use of AI in grading and assessment at Cornell must meet and adhere to. As AI tools, norms and practices in the educational environment continue to evolve, these minimum requirements may be reviewed and revised by approval of the AI Education Working Group, and relevant leadership and shared governance bodies as technologies, evidence, disciplinary norms and practices, and faculty and student needs and expectations develop and evolve.
Use of AI in Grading and Assessment at Cornell: A Proposed University-Wide Minimum Floor Policy
(updated November 2026)
Minimum requirements
Any use of AI in grading or feedback at Cornell must meet the following minimum requirements around responsibility, transparency, the right of human appeal, and security and data privacy.
Responsibility. Instructors retain responsibility for the soundness and accuracy of all grades and substantive feedback. Responsibility requires that faculty expertise and judgment remain in the lead of assessment. Simple approval of AI-generated evaluations alone, without additional efforts at validation or verification, is unlikely to meet this standard. Where AI plays a role in assessment, the instructor commits to:
o setting and reviewing the criteria AI applies;
o reviewing AI output against actual student work;
o retaining the ability to explain and defend each grade without relying solely on AI’s reasoning; and
o making final evaluative judgments and the assessment and assignment of grades..
Before any use of AI to assist the evaluation process, instructors should validate its output against their own independent evaluation of a sample of anonymized student work, and repeat that check when the assignment, rubric, or AI tool changes. Care should be taken when assuming model proficiencies – one model may work well for one assignment while performing poorly on another. Similarly, per research cited below, model performance may vary across model versions or generations, and test/retest reliability may be weak.
Any TA use of AI in grading must meet the full set of responsibilities articulated in this document, and must be fully disclosed to, reviewed and approved by the course instructor. The decision to use or not use AI in assessment remains fully under the responsibility and authority of the instructor.
Transparency. Any use of AI in grading or feedback must be clearly disclosed to students before the assessment occurs, for example by clear disclosure in the syllabus or assignment instructions. It should also be disclosed to leadership – for example, the relevant department Chair or Director, as defined by the course parent. Department leadership retains the right to disallow particular uses of AI in assessment where these violate university policy and/or disciplinary or departmental norms and agreements.
Right of human appeal. In cases of dispute, students retain the right of appeal and reassessment by members of the course teaching staff. Instructions and procedures for appeal should be clearly shared with students, for example through a clear statement on syllabus or in assignment instructions.
Security and privacy. All faculty are responsible for the security of grades and private student information, including responsibilities under FERPA. Only Cornell-approved apps and instances (e.g. university licensed and supported AI tools and secured storage appearing on Cornell’s regulated data chart) may be used. No use of AI in grading is permitted where the security and integrity of student information cannot be verified and maintained. Faculty may reasonably conclude that the risks of exposing student work to AI systems outweigh the potential benefits. That is a legitimate reason not to use AI in the assessment process..
Use of AI in Grading and Assessment at Cornell: A Proposed University-Wide Minimum Floor Policy
(updated November 2026)
Department and college policies
Beyond this minimum floor, individual faculty, departments, and colleges may adopt additional policies, including restrictions, on the use of AI in grading and feedback. These should reflect course, disciplinary, department, or college needs and continue to meet the requirements of responsibility, transparency, appeal, and security above. Collective policies at the department or college level should be clearly communicated to faculty, students, and the relevant college leadership. In making these determinations, department and college leaders are encouraged to consider norms, values, and student and faculty needs specific to their disciplines and fields; and Cornell’s Core Principles in AI and Education.
Beyond the minimum requirements, the following considerations can help faculty and departments decide whether and how AI may assist with assessment; or conversely, whether additional checks or restrictions are warranted beyond the minimum floor conditions above. These are given as suggestions for potential awareness and consideration, and are listed in order of importance.
1. Nature of the assessment. Current research shows that AI is better suited to objective, highly structured tasks with clear criteria or a correct answer, such as computational problems or programming exercises. Even there, an answer key or test suite (or automated code checker) may be more reliable. Greater caution is warranted where assessment requires interpretation, creativity, judgment, ethical reasoning, contextual understanding, or evaluation of complex and ambiguous intellectual processes. AI-assisted grading may be entirely appropriate in some contexts and inappropriate in others, regardless of technological capability.
2. Degree of evaluative judgment. Summative evaluation warrants particular care because its consequences for students are greater. Formative feedback, however, can have profound effects on learning trajectories, confidence, revision practices, and intellectual development. The need for human involvement should depend on the degree of evaluative judgment involved, whether or not the work contributes to a grade, and how tools have been vetted prior to use.
3. Equal standard across course sizes. Class size is a resource consideration. Students in large courses should not receive a fundamentally different standard of evaluation than students in small courses. If meaningful human evaluation is necessary to assess student work, that holds regardless of course size. Scale may shape implementation strategies, but not the educational principles underlying assessment.
4. Stakes. Low-stakes and high-stakes assignments may warrant different levels of review.
5. Discipline. Disciplinary norms may shape what counts as quality work and who is qualified to evaluate it.
6. Student level. Needs and expectations may differ from first-year to doctoral work.
7. Modality. Text, mathematics, code, and audio or video work may raise different questions and need to be assessed differently according to college or department needs. .
8. Augmenting vs. replacing. AI that helps an instructor give feedback differs from AI that substitutes for the instructor’s judgement and evaluation. Cases where AI can help strengthen the quality, timeliness, richness and efficacy of student feedback and learning, while meeting the minimum standards above, may be considered and supported as potential sites of experimentation
Use of AI in Grading and Assessment at Cornell: A Proposed University-Wide Minimum Floor Policy
(updated November 2026)
and pro-learning uses. Departments, colleges, and individual instructors are encouraged to evaluate such claims according to discipline-specific norms, values and needs.
9. Tool variability. Performance differs across models, versions, and general-purpose versus purpose-built tools. Performance in one course or assignment does not establish its performance in another, and models are routinely “nerfed”, or downgraded, without notice to users.
10. Teaching teams. The primary instructor must ensure that all members of the teaching team have the capacity to evaluate AI output critically, and have understanding of how the assessment tool was designed to work. Per note on TA use above, course instructors retain the ultimate responsibility for the use or non-use of AI in assessment across the course. This stipulation should be clearly communicated, discussed and reinforced with all members of the teaching team (for example, in documents describing course-level TA expectations, TA appointment letters, or potentially graduate handbooks).
What current research shows
Research through 2026 shows that performance and agreement between AI and human evaluators varies widely by task, model, and context. These findings may add and help to inform the college-, department- and instructor-level considerations above. Research in this area continues to evolve and is being tracked by Cornell’s Center for Teaching Innovation. Additional references are in the appendix.
Agreement varies widely. A synthesis of 65 studies (2022 to 2025) found that agreement between AI and human essay scores varied substantially across and within studies, with many values below the accepted benchmark for automated scoring (Li et al. 2026).
Consistent scores vs “valid “ones. In authentic university writing, AI agreed moderately with one experienced human rater and not at all with another. AI captured a narrower notion of quality than human raters (Kerwat et al. 2026). Kerwat et al recommend that LLMs “may be useful as supervised assessment-support tools, but they are not yet ready to be used independently for marking higher education student writing.” Further, a model can return nearly identical scores across repeated runs and still differ systematically from expert judgment (Okyay 2026, preprint).
Bias. Scores can shift with features unrelated to the quality of the work, including first-language background and demographic cues inferred from writing style (Yang et al. 2025).
Structured tasks. AI performs better where criteria are clear or a correct answer exists, but results still vary across models in technical courses (Saez et al. 2026). Where an answer key or test suite exists, it is often the more reliable tool.
Manipulation by prompt injection. Findings are mixed, but vulnerabilities exist. Instructions hidden in student submissions raised AI-assigned grades across all six widely used tools tested (Milani et al. 2025). Susceptibility varies by model: one large study found small effects on frontier models and large effects on a smaller one (Wanjura et al. 2026).
Feedback and trust. Students rate teacher feedback as more helpful and more trustworthy than AI feedback (Henderson et al. 2025), and rate feedback lower once told it was AI-generated (Nazaretsky et al. 2026).
Use of AI in Grading and Assessment at Cornell: A Proposed University-Wide Minimum Floor Policy
(updated November 2026)
References cited:
Okyay, Savaş. 2026. “Beyond Mean Scores: Individual- and Trait-Level Agreement Between Human and Generative AI Raters in EFL Writing Assessment.” Research Square. https://www.researchsquare.com/article/rs-10728298/v1 [not yet peer-reviewed]
Henderson, Michael, Margaret Bearman, Jennifer Chung, Tim Fawns, Simon Buckingham Shum, Kelly E. Matthews, and Jimena de Mello Heredia. 2025. “Comparing Generative AI and Teacher Feedback: Student Perceptions of Usefulness and Trustworthiness.” Assessment & Evaluation in Higher Education. https://doi.org/10.1080/02602938.2025.2502582
Kerwat, Logayna, et al. 2026. “Can Large Language Models Reproduce Higher Education Grade Bands? Cross-Model Study of Calibration and Grading Bias in Authentic Student Writing.” Assessment & Evaluation in Higher Education. https://www.tandfonline.com/doi/pdf/10.1080/02602938.2026.2734796
Li, Hongli, Che Han Chen, Kevin Fan, Chiho Young-Johnson, Soyoung Lim, and Yali Feng. 2026. “Agreement Between Large Language Models and Human Raters in Essay Scoring: A Research Synthesis.” Forthcoming in Building Knowledge of Generative AI in Applied Linguistics. Preprint: arXiv:2512.14561
Milani, Alfredo, Valentina Franzoni, Emanuele Florindi, Assel Omarbekova, Gulmira Bekmanova, and Banu Yergesh. 2025. “When AI Is Fooled: Hidden Risks in LLM-Assisted Grading.” Education Sciences 15 (11): 1419. https://doi.org/10.3390/educsci15111419
Nazaretsky, Tanya, et al. 2025. “Who Gives Feedback Matters: Student Biases Towards Human and AI-Generated Formative Feedback.” Journal of Computer Assisted Learning. https://doi.org/10.1111/jcal.70153
Saez, Y., Garcia, L.M., Mochon, A. et al. Evaluating large language models for AI-assisted grading: a framework and case study in higher education. Sci Rep 16, 18035 (2026). https://doi.org/10.1038/s41598-026-48656-3
Wanjura, Benjamin, Dan Shapiro, Ethan R. Mollick, Lilach Mollick, and Lennart Meincke. 2026. “Prompting Science Report 5: This Is an Excellent Paper: The Effects of Prompt Injection on Grading.” SSRN. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=6510758 [not peer-reviewed]
Yang, K., M. Raković, D. Gašević, and G. Chen. 2025. “Does the Prompt-Based Large Language Model Recognize Students’ Demographics and Introduce Bias in Essay Scoring?” In Artificial Intelligence in Education (AIED 2025), 75–89. Springer. https://doi.org/10.1007/978-3-031-98417-4_6
Use of AI in Grading and Assessment at Cornell: A Proposed University-Wide Minimum Floor Policy
(updated November 2026)
Appendix: Additional References
Disclosure: AI tools from Anthropic (Claude) and Elsevier (Leapspace) were used to support the search for relevant research as described below; analysis and summarization was performed by members of the Center for Teaching Innovation staff .
Purpose of assessment
Sadler, D. Royce. 1989. “Formative Assessment and the Design of Instructional Systems.” Instructional Science 18: 119–144.
Tai, Joanna, Rola Ajjawi, David Boud, Phillip Dawson, and Ernesto Panadero. 2018. “Developing Evaluative Judgement: Enabling Students to Make Decisions about the Quality of Work.” Higher Education 76: 467–481.
Reliability and validity of AI grading
“Beyond Mean Scores: Individual- and Trait-Level Agreement Between Human and Generative AI Raters in EFL Writing Assessment.” 2026. Research Square. https://www.researchsquare.com/article/rs-10728298/v1 [not peer-reviewed] [verify authors]
Bui, Ngoc My, and Jessie S. Barrot. 2025. “ChatGPT as an Automated Essay Scoring Tool in the Writing Classrooms: How It Compares with Human Scoring.” Education and Information Technologies 30: 2041–2058. https://doi.org/10.1007/s10639-024-12891-w
Flodén, Jonas. 2025. “Grading Exams Using Large Language Models: A Comparison Between Human and AI Grading of Exams in Higher Education Using ChatGPT.” British Educational Research Journal 51 (1): 201–224. https://doi.org/10.1002/berj.4069
Kerwat, Logayna, et al. 2026. “Can Large Language Models Reproduce Higher Education Grade Bands? Cross-Model Study of Calibration and Grading Bias in Authentic Student Writing.” Assessment & Evaluation in Higher Education. https://www.tandfonline.com/doi/pdf/10.1080/02602938.2026.2734796
Li, Hongli, Che Han Chen, Kevin Fan, Chiho Young-Johnson, Soyoung Lim, and Yali Feng. 2026. “Agreement Between Large Language Models and Human Raters in Essay Scoring: A Research Synthesis.” Forthcoming In Building Knowledge of Generative AI in Applied Linguistics. Preprint: arXiv:2512.14561
Mendonça, Pedro C., Filipe Quintal, and Fábio Mendonça. 2025. “Evaluating LLMs for Automated Scoring in Formative Assessments.” Applied Sciences 15 (5): 2787. https://doi.org/10.3390/app15052787
Pack, Austin, Alex Barrett, and Juan Escalante. 2024. “Large Language Models and Automated Essay Scoring of English Language Learner Writing: Insights into Validity and Reliability.” Computers and Education: Artificial Intelligence 6: 100234. https://doi.org/10.1016/j.caeai.2024.100234
Use of AI in Grading and Assessment at Cornell: A Proposed University-Wide Minimum Floor Policy
(updated November 2026)
Saez, Y., L. M. Garcia, A. Mochon, and P. Isasi. 2026. “Evaluating Large Language Models for AI-Assisted Grading: A Framework and Case Study in Higher Education.” Scientific Reports. https://doi.org/10.1038/s41598-026-48656-3
Tang, Xiaoyi, Hongwei Chen, Daoyu Lin, and Kexin Li. 2024. “Harnessing LLMs for Multi-Dimensional Writing Assessment: Reliability and Alignment with Human Judgments.” Heliyon 10 (14): e34262. https://doi.org/10.1016/j.heliyon.2024.e34262
Wetzler, Elizabeth L., Kenneth S. Cassidy, Margaret J. Jones, Chelsea R. Frazier, Nickalous A. Korbut, Chelsea M. Sims, Shari S. Bowen, and Michael Wood. 2025. “Grading the Graders: Comparing Generative AI and Human Assessment in Essay Evaluation.” Teaching of Psychology 52 (3): 298–304. https://doi.org/10.1177/00986283241282696
Yavuz, Fatih, Özgür Çelik, and Gamze Yavaş Çelik. 2025. “Utilizing Large Language Models for EFL Essay Grading: An Examination of Reliability and Validity in Rubric-Based Assessments.” British Journal of Educational Technology 56 (1): 150–166. https://doi.org/10.1111/bjet.13494
Bias
Yang, K., M. Raković, D. Gašević, and G. Chen. 2025. “Does the Prompt-Based Large Language Model Recognize Students’ Demographics and Introduce Bias in Essay Scoring?” In Artificial Intelligence in Education (AIED 2025), 75–89. Springer. https://doi.org/10.1007/978-3-031-98417-4_6
Manipulation of AI graders
Milani, Alfredo, Valentina Franzoni, Emanuele Florindi, Assel Omarbekova, Gulmira Bekmanova, and Banu Yergesh. 2025. “When AI Is Fooled: Hidden Risks in LLM-Assisted Grading.” Education Sciences 15 (11): 1419. https://doi.org/10.3390/educsci15111419
Wanjura, Benjamin, Dan Shapiro, Ethan R. Mollick, Lilach Mollick, and Lennart Meincke. 2026. “Prompting Science Report 5: This Is an Excellent Paper: The Effects of Prompt Injection on Grading.” SSRN. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=6510758
Student perceptions of AI feedback and scoring
Henderson, Michael, Margaret Bearman, Jennifer Chung, Tim Fawns, Simon Buckingham Shum, Kelly E. Matthews, and Jimena de Mello Heredia. 2025. “Comparing Generative AI and Teacher Feedback: Student Perceptions of Usefulness and Trustworthiness.” Assessment & Evaluation in Higher Education. https://doi.org/10.1080/02602938.2025.2502582
Nazaretsky, T., P. Mejia-Domenzain, V. Swamy, J. Frej, and T. Käser. 2026. “ Who Gives Feedback Matters: Student Biases Towards Human and AI-Generated Formative Feedback.” Journal of Computer Assisted Learning 42, no. 1: e70153. https://doi.org/10.1111/jcal.70153
Thomas, Mackenzie L., Seyma N. Yildirim-Erbasli, and Shruthi Hariharan. 2026. “Exploring Undergraduate Students’ Perceptions of AI vs. Human Scoring and Feedback.” The Internet and Higher Education 68: 101052. https://doi.org/10.1016/j.iheduc.2025.101052

Leave a Reply