I had the exact same problem a couple years ago. I spent about $200 on ammo components trying different loads and bullets but nothing close to a 4" group at 100. Tried all the things mentioned here with no luck and velocities that were 350 fps less than they should be and capped out even with different powder charges. Put in a new stock and trigger and still no fix. Sent it back to Remington and they gave me a new barreled action. Since a return sounds like no option for you, just build off of it.