It sounds an awful lot like the recoil is bothering you and you're developing a flinch, probably starting to lighten your hold a bit and maybe even dropping your shoulder back in anticipation of same.
Think your way through the fundamentals and make dead sure you're really pulling it in tight before pulling the trigger. If you're shooting off of a bipod, lean into it a bit, this is called "loading the bipod".
The tighter the bond between the butt and your shoulder the less impact there is from the recoil. The lighter it is the more the rifle smacks you before then driving you back.
Another technique that works well for a lot of people is that instead of fighting the recoil by leaning in they just pull the rifle in tightly and roll with it, not fighting it at all.
Both work, the latter is harder to learn because the natural reaction is to brace for an impact but it basically allows your whole body to take the recoil rather than just the contact point so your body becomes a shock absorber.
When I was young 10 I developed a really bad flinch from shooting a 10g on a goose hunt and it took me years to get over it. Today, after a lot of work I flat never consider it at all.