When is the random order of answers set in stone?

Added by Stefan Reinsberg over 8 years ago

I'm on the eve of a multi-section exam (1000 students) and rather nervous - probably more than most of my students. I have printed the exams (there are individual copies each with 7 pages with randomized order in the answers, no shuffling of questions). I did this in batches for my three sections. I am somewhat concerned whether the marking is going to stay in sync with the randomization of question order.

At what point is this order fixed such that in the marking process the right answers are marked as correct. I know that at some point I have to stop fiddling with the tex file. But how about Layout detection? Also, if I use the command-line interface:

auto-multiple-choice prepare --mode s --prefix ./ midterm.tex --with pdflatex --n-copies=huge-number
is the layout detection, and storage of question ordering performed as well? Or is there an algorithmic dependence of question order on copy number?

Lastly, is there a convenient way for merging batches of user-improved mark recognition. I will have several markers scan and check the mark recognition on different work stations but ultimately would like to merge their efforts or do I have to merge the csv table exports later?

Thanks for your help - I'll promise feedback on how this large-scale (for me) experiment worked out.


Replies (5)

RE: When is the random order of answers set in stone? - Added by Alexis Bienvenüe over 8 years ago

is the layout detection, and storage of question ordering performed as well? Or is there an algorithmic dependence of question order on copy number?

Questions and answers ordering is done by the LaTeX package, at latex-compiling time. This ordering is random, but the randomness is controlled by an internal (defined in the automultiplechoice LaTeX package) pseudo-random generator that is seed with the same value at each run. So the ordering is the same at each time you run LaTeX.

Lastly, is there a convenient way for merging batches of user-improved mark recognition.

No. This could be implemented, but needs a little work.
Perhaps you can work as follows:
  1. After having printed the copies, give an archive of the project directory (tar.gz or zip) to each work station
  2. Each work station creates a project from the archive, with a different name identifying it.
  3. Give (1000 / n) completed answer sheet scans to each work station.
  4. Each work station does all the job with automatic/manual data capture, and manual correction of ticked-or-not boxes (zooms window)
  5. Each work station gives you the exported CSV file, and you merge them very easily.

RE: When is the random order of answers set in stone? - Added by Stefan Reinsberg over 8 years ago

Thanks - The merging at the final stage sounds like a workable solution. Presumably I can even move project directories from one workstation to the next and then continue working on it ...

This lovely automultiplechoice package has re-ignited my love for latex. Thanks for this great piece of software.

RE: When is the random order of answers set in stone? - Added by Alexis Bienvenüe over 8 years ago

Presumably I can even move project directories from one workstation to the next and then continue working on it ...

Yes: this will also work.

RE: When is the random order of answers set in stone? - Added by S. McKay over 8 years ago

Lastly, is there a convenient way for merging batches of user-improved mark recognition.

Perhaps I don't understand the question, but there is an easy way to do this, depending on what you want. I assume that you want to take separate scans of different sections and merge them into one file?

There is open source software (Mac, Linux, Windows) that will do what you want. It's called pdfsam (for pdf split and merge). There is a commercial version, but the open source version does all I need.

If in the other hand if each grader has a single page, I wrote a python script that merges pages back into exams (the pages all need to be in the same order). Email me if 6 are interested. However, I wouldn't think you would need that in AMC. (i use it for a different reason.)

RE: When is the random order of answers set in stone? - Added by Stefan Reinsberg over 8 years ago

Thanks for the reply. I didn't articulate my request clearly: With merging I meant that for one project I would like to have people on several workstations check the mark recognition for parts of the exam and then before the marking stage merge their efforts. Essentially this would be a merging between 'Data capture' and 'Marking'.

I know that I can merge PDFs (pdftk etc) or even multi-page tiffs but this is a merge on the database level. I believe Alexis' suggestion is just as good in that I will merge at the end of the whole process. In essence I will have multiple isntances of a project and each marker will only work on a fraction of all the scans and hence only mark that entire fraction. The merge happens with the exported CSVs.

(1-5/5)