From the target pics you posted the 100 yard group looks to be about .75" right on average. If you were truly zeroed at 100 then the group would be at the center. At 100 yards it won't seem like much but the difference of vector at 400 yards along with spin drift or magnus effect depending on barrel twist rate could certainly push poi 4 or 5 inches right. I like to have my 100 zeros truly at zero or left of zero to mitigate this issue. Others have mentioned changing shooting positions and checking scope tracking etc. which are all good suggestions. I'd just like to suggest verifying your 100 yard zero is either exactly at, or left of center before moving out to 400 again. Good luck and keep us posted.