availability is going to be a big factor too. If you get Savage, they make a wide variety of styles and most shoot very well out of the box. Well priced too. With a growing aftermarket, they can be worked up as easily as a Rem 700.
I had a 6BR based on a Savage action and it was one of the most consistent and accurate rifles I have ever had the pleasure of owning/shooting.
Howas, Win make very nice rifles too. I build a lot on surplus actions and favor the Enfield P14/17 actions. Very strong and surprisingly very accurate. Lots of parts to convert to modern function.
Of course, the M98/M48 Mauser is also a possibility. Many were converted to BR rifles before "modern" actions were readily available. I have owned and shot others that would all do 1/4 to 1/2 MOA. Certainly good enough to pop a varmint at long range.
Enjoy your quest...
Jerry