Are you crimping? How much jump to the lands do these loads have? How much bullet is in contact with the neck of your case? What's your brass prep method? What brass are you using?
Issues with stability aren't always due to twist rate, especially if it's an inconsistent/intermittent issue. If some rounds are experiencing excessive yaw in flight, that can result in bullets impacting at angles. If that's your issue, you have to start troubleshooting to find the actual cause.
Because you're not experiencing issues with other bullets, I tend not to think it's an issue with your crown or even twist rate. You need to start ruling out things though, one at a time, so verifying your twist rate is still something I'd recommend.
Sure, you can just stop using this bullet and try another, but ultimately that's not helping you figure out your actual problem so you can prevent it in the future. That's obviously up to you though.