Maybe what's holding you back is that 0.1" thing you're not doing on your handloads, or that other 0.1" thing you're also not doing. Or both of those things combined for 0.2" that's holding you back? How would you know? It doesn't take hundreds of rounds to test. None of us can perform 100% all of the time, but don't you ever fire a shot that felt right but it didn't impact where it was supposed to and you think hmmmmm... You'll never know if it was you vs the ammo you leave a little load development on the table untested.
I definitely understand what you're saying.
But to legitimately prove a decrease in group size it can literally take hundreds of rounds, depending on the repeatability of the system. I understand the thought if having two 5 shot groups side by side ones a .5 and ones a .4, and then picking the smaller one that had a .003 seating difference and considering it a change. Then taking that and doing a charge weight test and you have one 5 shot group land at .4 and one at .3. So you pick the .3 and now you've "made" a .2 change in the accuracy of the load.
But I just shared two large sample size groups, much more statistically relevant than a 5 shot group (and most people don't even shoot 5 shot groups) and there is a 40% variance with the exact same load. And a rifle that shoots 12 rounds into 1/2 MOA at 100 yards is certainly an accurate rifle to base data off of.
So of course if you're going to go through all the hoops of testing, you're going to pick the smaller group from each test. But unless you're going to establish a legitimate baseline accuracy for the initial load (which is hardly ever done with any significant amount of shots) all the tweaking and tuning from the testing afterwards have virtual nothing for a comparison. You just think you're progress based of off virtually nothing but an initial 3 shot group normally.
And I'm not saying YOU personally. But that is the case a lot of times.
In the video I shared. The F class shooter literally did an entire meticulous seating depth test, as he would normally do. He then picked the absolutely worst depth he initially found and the best depth he found. He shot them for an actual large sample size (33 rounds a piece) and the initial worst small sample group outperformed the best small sample group.
So it really can take a significant amount of shots to prove without a doubt that you've done something to increase your accuracy.
It's not like we're taking 2 MOA guns and taking them to 1/2 MOA. Thats an apparent and obvious change. But taking a .5 MOA gun to a .4 MOA gun is extremely hard to prove and repeat.