i know the hunter is only 4 oz heavier, I'm just trying to get my build where i want it.
I think it comes down to how you intend to use this rifle, and it kind of sounds like you have just about the same goal that I had when I built my 338.... a true long range hammer for hunting that is still light enough to carry all day. Those two goals kind of contradict each other so you've got to strike a balance.
A build like C.O. Shooter's is really going to maximize the capability of a big 338, that's why most guys build them a little heavier. Personally, that is just more weight than I care to lug around the mountains, so i sacrificed some range capability for less weight. But to the ranges I'm going to hunt, I've got supreme power and accuracy in a weight that is reasonable to backpack hunt with.
So I think you're on the right track with not wanting the extra 4 ounces with the hunter action ... that's a 1/4 pound! I really wouldn't worry about the flex in either action... I've never owned a defiance action, but from what I've read about them they are pretty much as good as it gets and they're all going to be accurate.
I think Bigngreen gave the best advice of the UL action limiting your scope position on top of the action. And along those same lines, the integrated scope rail forces picatinny scope rings which then limits how low you can mount a scope, which then drives the need for a bigger stock to get the comb up higher... each one of those components adds some ounces, and those ounces all add up to pounds.
I'm just splitting hairs to make the point that you need to be very picky if you have a weight goal in mind, I really think those defiance actions are engineering works of art... You just made the comment that you've had some previous builds that you were not completely happy with, so I'm throwing in the voice to keep things light weight and be confident that the rifle will still be plenty accurate.
Now some others can chime in to go heavier to keep your build in check. Nobody is wrong, just remember each choice comes with a tradeoff.