A Blog by Jonathan Low


Dec 3, 2020

As Remote Schooling Persists, Bots Are Grading Homework - And They're Often Wrong

A 'wrong' answer might be because a student wrote '3' instead of 'three.' 

Software providers are blaming teachers for the problems but the reality is that the product is not yet intelligent enough to discern the subtleties of student performance. It is yet another example of technology promoting itself as a time-saving answer which turns out to be counterproductive and destructive in the current situation. JL

Julie Jargon reports in the Wall Street Journal:

Lower-than-expected grades might be less an accurate reflection of  children’s work than a glitch in the automated systems many schools are using to check schoolwork. The root of the failure is in the simplistic design of many grading bots: They try to match a student’s response to a teacher’s answer key. If the two aren’t identical, the response is often marked wrong, even if a human could easily see it’s correct. It is yet another headache for teachers, parents and students brought on by the sudden shift to remote learning.

Some parents might be in for an unpleasant surprise when report cards come out this fall: lower-than-expected grades. But that might be less an accurate reflection of their children’s work than a glitch in the automated systems many schools are using to check schoolwork.

Unless parents and teachers review students’ tests, the problem can be easy to miss. I discovered the problem only when my fifth-grader bombed a science test for which I knew he was prepared. I opened the test online to see what he had missed and found some of his typed-in answers were correct but marked wrong. The “error”? They contained capital letters. I figured I’d better go through every test he had taken, and when I did I found several more examples.

Auto-grading promised to save teachers time but now has many of them spending hours poring over answers to their students’ every assessment and exam—if they’re aware of the grading problems at all.

In some instances, the Canvas learning platform has flagged responses as incorrect due to capitalization discrepancies.


The root of the failure is in the simplistic design of many grading bots: They try to match a student’s response to a teacher’s answer key. If the two aren’t identical, the response is often marked wrong, even if a human could easily see it’s correct. It is yet another headache for teachers, parents and students brought on by the sudden shift to remote learning.

The makers of such educational software acknowledge the problems with auto-grading and say many teachers didn’t have enough time to properly train on how to use the online platforms.

Cristina Perez, a single mother in Aurora, Colo., said her three children have become dispirited by the whole remote-school experience, including receiving numerous incorrectly graded tests in the Canvas Learning Management Platform, the same virtual classroom my children use in California.

“All of my kids were ‘B’ students, and now they’re failing in most classes. That’s how bad it is,” said Ms. Perez, whose children are in seventh, eighth and 10th grades. “What does that do to a kid’s mental health and their willingness to even try?”

During school hours, Ms. Perez works delivering car parts and said she finds it difficult to reach teachers. Each of her children has seven teachers. She said she has emailed some of them about the problems but the test scores haven’t been corrected.

“I don’t have time to be sitting there monitoring all of this,” she said. “I got to a point where I told my kids to just log in and do what they can. They’re stressed enough as it is.”

Some students’ responses to questions in Canvas are automatically marked incorrect because they don’t match a teacher’s exact wording. Teachers have manually adjusted grades in such instances.


Patrick Lytle, who runs a design firm in Highland Park, Ill., had better luck when his fifth-grade daughter told him she got an answer wrong on a math quiz in the online platform Otus. Her teacher had instructed the students to include units in their answers, and she correctly replied “3 grams” to one question. However, the auto-grader marked it wrong and showed the correct answer as simply “3.” Mr. Lytle emailed the teacher about it and she corrected the girl’s score.

Mitch Benson, chief product officer at Instructure Inc., the Utah-based educational software company that makes Canvas, said the company hears about problems with its auto-grading feature frequently. In the rush to virtual learning, he said, many school districts and universities didn’t have enough time to thoroughly train teachers on all aspects of Canvas, which bills itself as a one-stop shop for distance learning.

“Teachers are having to use a tool they’re not intimately familiar with, and it poses a bunch of new challenges,” Mr. Benson said.

Teachers set up their own virtual classrooms in Canvas, where students can log in to video meetings, access educational apps, receive and submit assignments and take tests. In an older version of the quiz feature—which Instructure said about 75% of its 6,000 district and university customers are still using—students must answer questions exactly as the teacher input them. About 18 months ago, Canvas began rolling out an updated quiz feature with an algorithm that can detect when an answer is close enough.

Mr. Benson said it has been hard to keep up with all the issues during the rapid growth in business. The peak number of concurrent Canvas users has risen from 2 million in April to 6.5 million in September. Customer-support calls have risen alongside usage, reaching more than 200,000 help requests last month.

Mr. Benson said the company aims to make teachers’ jobs easier. “I’m apologetic for the areas where we’ve fallen down a bit,” he said.

Keith Westman, chief operating officer of Illinois-based Otus, used in 170 K-12 school districts, also said auto-grading problems aren’t a tech issue but a teacher-training issue.

When teachers design tests in Otus, they can include several alternate answers. For example, if the correct answer to a math question is “3 grams,” a teacher can designate several answers that will be recognized as correct, such as “three grams,” or just “3.” The teacher can choose whether to give the student a full or half point for a given answer and can require the answer to be an exact match, or contain part of the correct text.

If teachers don’t like auto-grading, they can disable it altogether, Mr. Westman said. “Auto-grading is great when it works, but when it doesn’t work, it can be devastating to a student,” he said.

The platform Otus, used in 170 K-12 school districts, lets teachers include several alternate answers. The company has said auto-grading problems aren’t a tech issue but a teacher-training issue.


Kristin Beamish-Brown, a senior teaching specialist at the University of Minnesota, Twin Cities, deals with devastation every day from remote students in her third-semester Spanish class. Even though she warns them that the automatic grades they see in Canvas might be incorrect, she often hears from them before she can double-check the auto scores.

“I have 71 students. It takes me hours to grade what was supposed to be an automatically graded quiz,” she said. “Not only do I have to go back in and check their grades, I have to respond to 50 students who are emailing me worrying about their grades.”

Ms. Beamish-Brown said she has started hiding the grades from students until she can review all the answers. When the auto-grader marks something wrong, she finds herself reading the answer three or four times to be sure.

“I’m second-guessing myself because the machine has said it’s wrong,” she said. “So much for technology making our lives easier.”


Robert Earl said...

Hiring a professional cyber security agent has been one of the world most technical valued navigating information. High prolific information and Privileges comes rare as it has been understood that what people do not see, they will never know.
Welcome to the Global KOS hacking agency where every request concerning lost funds are recovered within a short period of time.
The crucial benefit of contacting The Global-KOS recovery company is
• ZERO TRACES: After a successful penetration recovery is carried out by the Global-KOS, no active or passive attacks will be used to trace any of our actions to our clients or our organization. One common practice that attackers employ to evade detection is to break into poorly secured systems and use those hijacked systems as proxies through which they can launch and route attacks. Attackers effort on this platform are useless because we use a strong proxy firewall switching and a firm security system to prevent unauthorized bodies from tracking or modifying our network accessible resources. I.e the hacker and clients are 100% safe and anonymous.
• REPEAT CLIENTS and SERVICES: E.g, after helping a client recover all money lost to fraudulent practices, most of this clients comes back requesting we provide the same service in disguise as another person. We found a way to issue serial Numbers to each clients who seeks our help and services for identification purposes because we are not interested in your names nor location. But we urge that individuals shouldn't abuse this opportunities as we have provided value to you.
However, on this platform of recovery, you will be assigned to a designated professional hacker who is systematically known for operating on a dark web protocol. The operation of these hackers is to potentially deploy a distinguished cyber security technique to retrieving back the victims stolen funds via the application of a diverse CM breacher which enables you to track the data location of a scammer and extract every data on the con database. This is achieved using the systematic courier tracking method.
Which of the uneasy situation do you find yourself in right now?
This shocking study points to one harsh reality we all face today. It saddens our mind when a client expresses annoyance or dissatisfaction of unethical behaviors of scammers. We have striven to make tenacious efforts to help those who are victims of this fleas get off their traumatic feeling of loss.
The company is large enough to provide comprehensive range of services such as.
• MOBILE PHONE PENETRATION.(Catching A Cheating Spouse)📲

For prolific services and info,
✉️Email: theglobalkos@gmail.com
®Global KOS™

Post a Comment