See attached files for a starting point, using development version of AMC.
For each question, I created two AMC questions (hiding the "Question" header for the second one): the first for the confidence level, and the second for the question itself.
You have to adapt the code for questionmult, depending on your scoring strategy.

Dear Alexis,

many thanks for the quick reply, that's a great starting point!

I'm using version 1.3.0 from AUR on Arch. Does the development version include any additional features needed here or should this work with the version I use?

Trying the example I get:

It does pick up the confidence level, but doesn't seem to calculate a mark for the actual question. No matter which box I tick it returns a 0 mark. So, I do wonder if setglobal.SCOREOK and setglobal.SCOREWRONG don't work properly. If I replace them with +1 and -1 in \scoring for the actual question I do get those values back. It seems that both default to 0 in my case and are not passed from the confidence part to the actual question. Unfortunately, I'm not enough of an expert on the inner workings of AMC to debug this. Is there anything I can do to try to trace why the values don't get passed through?

All the best,
Marc

I'm using version 1.3.0 from AUR on Arch. Does the development version include any additional features needed here?

Yes. The key point is the use of the setglobal.XXX directive, that was recently introduced with revision hg:1815c3151934. See #273.

Ah! I have now applied the diff to 1.3.0 and it sort of works...

The mark values for a correct answer at certainty low (0) and high (2) seem to have been swapped. Am I doing something wrong here?

Also, the layout of the answer sheet isn't great:

Is there a way to influence the layout and have the answers first and the certainty levels after them in one line like

Question 1: |A| |B| |C| |D| |E|    |L| |M| |H|

The mark values for a correct answer at certainty low (0) and high (2) seem to have been swapped. Am I doing something wrong here?

I'm afraid that setglobal.XXX was not correctly tested: this is a bug, that should be fixed with hg:2f5ae49dfa71. Can you test it?

I'm afraid it is not possible to get confidence level after answers, but you can get a single-line layout with (right after the \def\AMCformQuestion#1{...} line)

\def\AMCformBeforeQuestion{\ifnum\value{AMCquestionaff}<0\hspace{1em}\else\vspace{\AMCformVSpace}\par\fi}

Brilliant, this all works now as far as I can tell!

I'm attaching my latest test template for future reference for others that might want to try this marking approach too and the original research paper "Confidence-Based Marking - towards deeper learning and better exams" by A.R. Gardner-Medwin, Dept. Physiology, University College London.

Many thanks Alexis for the quick responses and bug fixes!

All the best,
Marc