I have tried the same test may times with my various competition rifles, I usually start at .010" jam and move backwards with 3 shot groups. When I find the best OAL, I write it down and begin a new sequence with another powder and primer.
PS - I rely on marks on the bullet in my dummy round. The bullet comparators are not consistent.
Personally I don't think 3 shots groups tell you enough, should be 4 minimum but 5 is ideal.
If you shoot 3 & have 2 touching & 1 flyer then the result is inconclusive imo, shoot 4 with 3 touching & 1 flyer then its worth following.
Also I dont get the statement in red, do you mean you cannot get a consistent result by using a comparator or the comparator's themselves are not consistent??
How do you then know what measurement you are at??
I have been using Hornady comparators for 12 years & never had a problem with them!!
I use the anvil which I find gives a better result as it keeps the base of the case more level.
You wont get marks on the projectile at 10 thou anyway!!
Anyway each to their own, reloading is about whatever works for you!
So my process is-
Firstly I obtain my lands measurement for the projectile I am using with a Hornady OAL gauge, the modified case I use is a case that is fireformed to my chamber & then reamed to fit the OAL gauge, measure CTBO 5 times to get an average.
I start all my LD @ 10 thou off lands, load 4 or 5 rounds per powder increment, sometimes 1gr intervals if the min/max range is 5gr or 1/2gr loads if there is only say 3gr min-max load.
I then choose the best group from charge range.
My next step is to change the CBTO to 5thou, 15thou & 20thou with 4 each.
Find the best from that & then if I want to try & bring it in more or didn't quite get the result I was looking for I will then do some smaller 1/2gr powder increments either side of that charge with the best performing CBTO.
By that stage I have fired 40 rounds or less, if I am still not happy I will go through the same process with a different powder.