Yah, I had the same rifle in a 300 RUM that was ridiculously accurate for an out of the box rifle with nothing more then a trigger job. NO bedding, NO floating, NO lugs lapped, NO truing of the action. Just a trigger job, and some hand loads it liked, and it was a 1/2-3/4 MOA rifle out to 800 yards, and would ring a 15" gong at 1K very regularly.
Most rems will shoot at least 1 MOA with trigger job and fed what it likes, most will usually do much better then that.
If trigger job and reloads dont cut it, then maybe you can free float and bed it. If that dont work, your bout out of options. At least you'll have a decent action to screw on a custom barrel though!!