Issues with gradebot

I’ve completed the topics_quiz (Unit 3), and when I run the launch file for the quiz I see the robot accomplishes it’s task of entering the wall opening. I’ve run this multiple times without any issues, with the robot always landing within the target square. So I submitted the code for grading, and I was very frustrated when the results were a 8/10, the test says it failed the last part. The testing dialog says that it would give the user details regarding why it failed. More frustration since the “details” were a list of a number of things that could have gone wrong (didn’t finish is 90 seconds, or didn’t go through opening, or crashed into wall) that doesn’t help at all.

Since the test never failed when I ran it, and the “clues” were not useful I needed to guess why my robot failed the last step.

  • First run, fails last step, don’t know why, was it a crash, timeout, or miss target. It’s hard to troubleshoot without information.
  • Second run, I assume maybe it’s taking too long, so I speed up the motion of the robot. Re test several times on development, still working 100%. Re-submit, still fails on last step.
  • Third run, still no clue, so I adjust some of my conditional statements to make the tolerances tighter, to make sure I’m right on the target mark (which isn’t marked in the simulator, more frustration). Still fails last step.
  • Fourth run I increase my timer frequency so that I have tighter resolution of ‘odom’ checks and reacting to that data. My testing still works 100%, so I re-submit. Now I fails during confirming reading of odom data! And I get a 4/10. What? Why? I increase the rate at which I read and use the odom data and now gradebot says I’m not even reading it! By now my head is exploding, I’ve run this thing countless times without fail, but gradebot disagrees.
  • Fifth, final run, Ok I don’t understand why gradebot is failing me, but I know my first run earned an 8/10, so I revert all my code to that state. I re-test myself again a few times without any failures. I submit, and wait… FAIL agian, on the odom step!! what, I have not changed how I read the odom data at all and now it’s failing for that. No, gradebot you are wrong!

Now I can’t see the solution, and I’ve got a poor grade of 4/10 for my first quiz. On a project that is not failing at all when I test this.

Very frustrated with gradebot. I’ve done good work, and I’m happy with the course thus far, since I am learning. But now I can’t see the solution, so I can’t learn what gradebot sees that I don’t see. How does gradebot test for final step? My robot is on the target position. Does it expect so logging message? I just stopped the robot at the position (sending 0 cmd_vel), was I supposed to exit the class somehow? How does gradebot calculate the time taken? Simulation time? Time from when launch started? How can I clock how long the robot took? What is gradebot looking for when I run my tests I see the robot stop at the target position. I’ve done all the requirements described in Grading Guide, yet gradebot still fails.

Now what? I’ve got a 4/10 on my first quiz, so I’m on track to fail the requirements for the certificate for this course because the stupid gradebot machine isn’t working. I don’t care that much about the certificate, I’m taking the course to learn, no for collecting certificates. But I would like the certificate once I earn it. So, now what? I’m concerned since now I have to average better than 8/10 on future quizzes to get my total average above 8 and gradebot won’t give out scores better that an 8.

Very frustrated, please advise what criteria gradebot uses to score these so that I can prepare for the future.

@mackcom
Welcome to the community!

I apologize for the glitch and the frustration that resulted. Checking our logs, I see that you indeed scored 8/10 at some point, only missing the final point. Checking the final point does not behave as expected sometimes, and we are working on improving it.

On that note, I checked your code and found that it achieved the objective of the quiz, so I have upgraded your score to 10/10.

Regarding the /odom part that was failing later, I’m yet to figure out why the grader couldn’t find it, because I can when I check manually (the grader must be tired or something :slight_smile:).

Should you run into this kind of issue again, please let us know immediately you use up 2 or 3 slots, instead of trying to slug it out with Mr. gradebot.

By the way…


  1. Your scores in the Quizzes are not taken into account for the certificate of this course. You will get a certificate if you complete and present the Real Robot Project mentioned right after the topics quiz. See this post for more information
    • The quizzes prepare you for what you will encounter in the Real Robot Project.
  2. Great storytelling there! Sounded like a drama to me :smiley:.

Update
Found and fixed a bug in the gradebot, which was causing it to check the last point too fast, not waiting for the robot to reach the final destination.

Thanks for reporting it.

@bayodesegun Thank you very much for taking the time to look into this. After posting I was concerned that I ranted too much, but I also wanted to provide context. Also, thanks for the clarification regarding certificate requirements. Now that I know quizzes are for our own preparation I won’t get as frustrated about gradebot, since it’s just trying to help (and not act as a gate keeper). And I’ll also reach out sooner if I get frustrated again.

Regarding the quiz, even tho I accomplished the goal I do have some concerns regarding my code. Since I used my 5 submits I was never able to see the suggested solution. I’m wondering if there is a way I can get a peek at the suggested solution so that I can contrast that with my solution? In an effort to further my understanding.

I’m glad you found the bug in gradebot, happy to know that my rant lead to something good.

I’m finding the rigor of the quizzes and project very helpful for motivating my learning. So, even tho gradebot can be frustrating I am very happy with the course thus far. Thank you for everything!

You’re welcome.

Caveat: this varies by course. Some courses consider the quizzes. You can find details on the course details page, as described in the forum link I shared earlier.

Yes, there is a way. I added one more attempt for you. You need to submit again to see the solution.

I’m glad to hear. Thank you for the feedback.

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.