Perhaps someone else can address the brass shavings on the bolt face...
In most cases, I see that with heavy loads. But, it could be accentuated by too much bolt thrust due to a slick chamber. If neither of those, then something is scoring the brass. And, it may or may not have much to do with your groups.
Check the fololowig:
- you already addressed the scope mounts/bases
- barrel is floated (nothing touching i.e. fingers, sling stud)
- action screws torqued correctly
- proper cleaning techniques (e.g. carbon fiber rod, bore guide)
- if it has a brake, try without the brake
- try a box of premium factory ammo
You sound like you've got experience. But, I'm amazed at how many variations you've tried. I'd step back and start with the basics.
It sounds like you only use quality components. So, I would focus on Norma Brass, RL22, GM215M primers, and 168 Bergers. (that combo works for me. or, pick a combination that's worked for others) Vary only the powder charge and, work up from min load to max with the ogive right at the lands using 2 shots each at half grain increments.
Pick the best 2 shot group surrounded by a couple of good groups that are also of similar elevation. In other words, some range of powder charge that seems stable.
Then, load/shoot 3 rounds at different seating depths holding the powder charge constant and adding .040" seating depths until you get to > .100.
Pick the best group, and load that group shooting 2 groups of 5 shots.
Now, start refining the powder charge +/- half grain in smaller steps again.
The objective is to change only one variable at a time and keep fine tuning.
There are a number of other methods. but, it sounds like you got frustrated along the way and started trying too many different components.
Hope something here helps!