Unable to process new set of scan data

Added by Clayton Spencer almost 5 years ago

I've used AMC to generate and process surveys of students in several classes. I did not vary the question or answer choice order and had AMC generate enough versions for the largest class of students I needed to survey. Then I printed out the correct number of surveys for each class and gave them to faculty. My plan was to scan the surveys in separately and then process them by class/course in AMC. It worked fine for the first class, but for the second class I selected "Automatic", chose the correct pdf file of scanned surveys for the class, the command window indicated that the multipage document was being separated and it showed that the individual pdf file pages were being converted into bitmap images. AMC completed the process without error, but still indicated that it hod no data. There were no items listed in the widow to allow me to zoom in and confirm the markings etc. Does anyone know how I can get this to work? Again, I'm using the same set of printouts for each class but within the set of printouts there are different AMC version numbers.


Replies (5)

RE: Unable to process new set of scan data - Added by Darija Tadin almost 5 years ago

Hi CLayton,
what exactly do you mean by "there being different AMC version numbers within the set of printouts"?
You could have made copies of your project before proceeding to data acquisition and continued from there, using each batch of scans in a separate project copy.
If you don't use multiple copies of the project: before starting with a new batch of scans, you should delete the already processed pages (this will remove all the corresponding data) by selecting the rows in Data Capture tab and clicking the <Remove> button.

RE: Unable to process new set of scan data - Added by Clayton Spencer almost 5 years ago

Hi Darki,

Thanks for the reply.

I'll try to clarify. Here's what I did...
I created a LaTeX file that included the survey questions - a combination of Likert scale questions and True/False items. The file included a section for students to mark their ID number as well as a box to write in their name. I set it so that neither the questions nor the answer choices were scrambled. The largest class that I needed to survey was 60 students, so I had AMC generate 60 pdf files. (This was done from a single AMC "project". I didn't think about coping the project for each separate class.) I then batch printed the correct number of files for each class and distributed them to the instructors. For example, class A had 36 students, so I printed out files 1-36. Class B had 55 students, so I printed out files 1-55. I then collected the surveys from each class and scanned them into pdf files. All the surveys collected from one class were scanned into a single pdf file. I selected the "automatic" button on Data Capture tab on AMC and located the pdf file for class A. After processing with the Marking tab and loading a class roster for class A, I had AMC export a report of the scores from the "Reports" tab. I then went back to marking in the "data capture" tab and selected "automatic". I browsed for the class B pdf file and selected it to open. AMC appeared to separate the pages in the multipage document and convert the pdf to bitmap images. There were no errors here, but the data in the "diagnosis" window was still from class A. I did select all of the item in that pane and selected "remove". The items were cleared from the diagnosis pane. I then selected the "automatic" button again and AMC appeared to process the class B file, but no items were displayed in the Diagnosis pane and there was still a red bar that indicated "no data."

Is there a way that I can still have AMC process the files from the other classes that I surveyed and scanned?

Thanks for any help you can provide.


RE: Unable to process new set of scan data - Added by Darija Tadin almost 5 years ago

Hi Clayton,
I'm not 100% sure that all the steps I'll propose are necessary (or that they are enough, for that matter), but this is what I would do:
  1. Make a copy of the current project, leave the original be and work all the subsequent steps ON THE COPY:
  2. go to the project's directory and:
    1. delete all the images in ./scans/ subdirectory (if any)
    2. delete all the images in ./cr/ subdirectory (./cr/page-*.jpg and ./cr/name-*.jpg, if any)
    3. delete ./data/association.sqlite
    4. delete ./data/scoring.sqlite
    5. delete ./data/capture.sqlite
  3. Crossing your fingers that what you now have is a clean project, just past the Preparation stage, awaiting Data capture and the steps following it, make a new copy of the project and proceed with the data capture of next class's scans using this newest copy.
  • make sure there exists ./data/layout.sqlite and that the timestamp of the file is what you expect it to be (the time when you did the Layout detection)
  • make sure there exists ./DOC-calage.xy, its timestamp very similar to layout.sqlite's
  • using command-line, try compiling the source.tex by cd:ing to the project's directory (using whichever of the copies -- this won't damage the project's files) and do the LaTeX compilation (of your source.tex) manually, using the same compiler as declared in Preferences/Project/LaTeX engine. This is to make sure that you have not inadvertently changed the project source file(s). If all is well, the compilation should create a source.pdf, an almost exact copy of your DOC-sujet.pdf, with the DRAFT stamp on every page being the only difference between source.pdf and DOC-sujet.pdf.

If any of the side-thoughts has failed, you have a(nother) problem.

If you still don't get through, make another clean copy of the project (following the numbered steps above), go tick the Help/Debugging checkbox, do the data capture and attach the debug log here; the message you are now reading is the limit of my abilities without either a crystal ball or a debug log ;-)


RE: Unable to process new set of scan data - Added by Alexis Bienvenüe almost 5 years ago

On ubuntu, maybe you are another victim of the imagemagick update bug? Have a look at RE: automatic data capture suddenly not working: "ERROR()....

RE: Unable to process new set of scan data - Added by Clayton Spencer almost 5 years ago

Apologies for the delayed reply.

The resolution for the ubuntu imagmagick update bug also resolved my problem. I edited the policy.xml file in the Imagemagick-6 directory and changed the rights="none" parameter to rights="read". That seemed to do the trick. I was able to select the "automatic" button, select a pdf file, and have AMC perform the data capture. Thanks to all for your help with this issue.