[Solved] Student automatic association not working (need to check log AND delete association.sqlite)

Added by Alex Alex 5 months ago

When grading in command line, I often have some problems with the student association. For example, in the debug output of auto-multiple-choice association-auto:
"[...] Incorrect association for code value "18": 2 instances"
This apparently results in most (but not all) student numbers not being associated.
This can be solved by for example removing one of the copies, or modifying the copy in GIMP. After that, the association works (as shown by the debug output, which contains no errors), BUT the association will apparently not be made available to the exporting and annotating phase.

The solution is to delete association.sqlite then rerun association and the following stages.

In other words, the sqlite seems to be put in a "stuck" state when there is an error, and will not recover by itself (I would consider this a bug).

I would also suggest giving the user some output (not just in the optional debug file), as well as setting the return code to something other than 0, to tell them what happened. As it is, I'm greping the debug file for that specific error message, and I don't know if I'm catching all the cases.

Note: This is with AMC 1.5.0~rc2 (r:c71170c9).


Replies (7)

RE: [Solved] Student automatic association not working (need to check log AND delete association.sqlite) - Added by Alexis Bienvenüe 5 months ago

BUT the association will apparently not be made available to the exporting and annotating phase.

This should not be the case. Can you prepare a minimal setup and precise instructions to reproduce this problem?

I would also suggest giving the user some output (not just in the optional debug file)

Two potential problems are reported in the debug file:
  • Code value CODE not found in students list: ignoring when a code found on an answer sheet cannot be found in the students list
  • Incorrect association for code value "CODE": N instances when a code CODE can be found on N (>1) different answer sheets

Maybe I can also make AMC show these messages on stderr?

However, this automatic association is often not completely successful (at least with me), so that some manual tweaking has to be made…

RE: [Solved] Student automatic association not working (need to check log AND delete association.sqlite) - Added by Alex Alex 5 months ago

Maybe I can also make AMC show these messages on stderr?

Yes, that would be perfect. I would also suggest setting the return code of auto-multiple-choice association-auto to 1 in that case.

Two potential problems are reported in the debug file:

Code value CODE not found in students list: ignoring when a code found on an answer sheet cannot be found in the students list
Incorrect association for code value "CODE": N instances when a code CODE can be found on N (>1) different answer sheets ??

Thanks, that helps. I also had the first error in my last batch, and it did not cause the problem I mentioned.

Thank you very much for reacting to this issue so quickly. I will try to produce a minimal setup to reproduce the problem, though I'm not sure how to make that small enough (and I cannot post copies publicly without also anonymizing them). Any hints on how to reduce the project files to a minimal (or at least smaller) setup?

RE: [Solved] Student automatic association not working (need to check log AND delete association.sqlite) - Added by Alexis Bienvenüe 5 months ago

Any hints on how to reduce the project files to a minimal (or at least smaller) setup?

Maybe build a test from scratch, with a minimal source file with one code and one question, then convert pages to jpg and tick some boxes with gimp, so that the problem occurs.
This kind of test would be perfect to be included in the source tree tests, to avoid regressions during future development after this bug fix.

RE: [Solved] Student automatic association not working (need to check log AND delete association.sqlite) - Added by Alexis Bienvenüe 5 months ago

I would also suggest setting the return code of auto-multiple-choice association-auto to 1 in that case.

This would not integrate smoothly with AMC Gui and tests, so I will add a --hard-fail option for that.
Update: see git:81069c3ac1

RE: [Solved] Student automatic association not working (need to check log AND delete association.sqlite) - Added by Alex Alex 5 months ago

Alexis Bienvenüe wrote:

This would not integrate smoothly with AMC Gui and tests, so I will add a --hard-fail option for that.
Update: see git:81069c3ac1

That sounds great, thank you very much.

I have attempted to create the minimal setup for the above problem. Instead this setup seem to be unable to do the association even without any association errors (no "OK" in association debug log at all, grade.csv has a bunch of "?" names). The archive contains:

- The .tex subject, named autoassoc.tex
- Most of the latex and amc intermediate files (.amc, .cs, .xy, config.tex)
- the scoring.sqlite and layout.sqlite in data dir
- student_names.csv
- copies_bad.pdf (where two students put the same student number), which should create a "incorrect association for code" error.
- copies_good.pdf (where students filled their number correctly), which should work but doesn't
- grading_script.sh, a bash script that does all of the pdf splitting, grading, association and export. It ends by displaying grades.csv, which shows if the association worked or not.

The script is simple, the lines with the amc commands should be easy to paste into a terminal if bash is not available.

The files in this archive were generated with the same method as the project in which my initial problem appeared. In particular, it really should work with copies_good.pdf. I will attempt later to dichotomize the source of the discrepancy, but I guess this is also a problem in it's own right with this finicky auto association.

autoassoc.7z (135.9 kB)

RE: [Solved] Student automatic association not working (need to check log AND delete association.sqlite) - Added by Alex Alex about 1 month ago

Thank you for your fix of my last problem (in the follow up post).

I have added this line to my "grading_script.sh":
auto-multiple-choice prepare --mode b --data ./data autoassoc.tex
With this, my initial problem is not visible in the minimal example I uploaded (switching from "copies_bad.pdf" to "copies_good.pdf" then rerunning the script is enough to get the association to work).

This was still very useful, as I didn't understand before that "prepare --mode b" could influence association; and maybe that will be the key to my other problems.

Here is the updated (corrected) minimal example if anyone needs it

autoassoc.7z (135.9 kB)

(1-7/7)