I don't know anything about the Vanguard series of rifles, but what type of recoil pad, what does the rifle weigh, and if shooting from the bench….are you sitting straight up or nearly so. A good deal of lean into the rifle can/will increase "felt" recoil.
And, does the rifle fit you properly? The length of pull can affect "felt" recoil. If the stock has been cut to better fit you…..was the stock angle at the butt altered? If so, that can affect felt recoil……usually, not for the better.
The 300 Weatherby while having a bit of recoil……shouldn't be as bad as you describe!
Addendum: A properly fitting rifle with a good recoil pad will help a lot. Also, when shooting from the bench, a PAST Recoil Shield or similar will help a great deal.
Personally speaking, I'd avoid the brake on a hunting rifle……but, that's just me!