Would not the dimensions between the groove bottoms and the outside profile also not be exactly the same from leade to muzzle in a non-fluted, solid barrel? While gun drilling a barrel blank fare thee well centers the hole in the blank's center, there has to be some small error due to the barrel blank not being perfectly homogeneous in its metalurgy.
I doubt even the most accurate barrels have the hole exactly down the exact physical center of the barrel. But I don't think that makes any significant changes to where the muzzle axis points for each shot as the barrel heats up. Putting 20 to 30 shots fired 30 or so seconds apart through 30 caliber barrels heats 'em up quite a bit, but accuracy doesn't seem to suffer going from ambient temperature to very hot for the last few shots. Accuracy's about 1/2 to 5/8 MOA at long range with good barrels so shot.
Perhaps gun drilling, reaming, rifling and lapping a hole in a barrel blank's easier to make more exactly centered than fluting a finished barrel such that each flute's exactly the same shape and position from the bore.