I take it the misses did not hit consistantly together? If they did that would show a clear sight in problem. If they were scattered, it could be several things that went wrong.
-shooting in conditions that are not suitable for long range shooting
-Rifle is not up to long range accuracy. Just because it shoots well at close range does not mean its a long range rifle.
-The shooting set up was not suitable for long range consistancy.
-Shooter got a case of "buck fever"
There is no shame in the latter but shooting big game at long range is a difficult thing. Many will get a rifle that will shoot well on paper or steel at long range but when its a critter in front of them it all changed. Its totally different for all of us comparing paper to big game.
You really need a very clear and direct run down list to mentally prepare yourself for each shot at long range. IF you follow the steps for every shot, it will help with the "Routine" of making a long range shot.
Also, your set up is critical. I missed a big whitetail at 650 yards because I did not have the set up of the rifle right. The problem was that I did not remove the sling from the rear of the rifle but instead wrapped it around the rear bag. At the shot, this torqued the rifle and I missed.
Luckily, the buck had no idea where I was and ran out to 750 yards and stopped again. By this time I had corrected the problem and and on the second shot, the bullet landed exactly where I wanted it to.
As far as shooting a doe with fawns, I do not have much to comment about that. Personally I would not do it. THere are generally plenty of dry does out there to harvest. But, it comes down to if this was a legal deer to shoot at and I assume it was. As it was, its hard for me to tell someone what they should or should not shoot at.
It does greatly complicate things when there are fawns in the area if your taking multipule shots and missing.
Depending on the amount of food available, most fawns would be fine without their mother but I would worry more about mule deer fawns then whitetail as they live in generally much harsher areas and generally have fewer food sources and more predators. Still, if it was a legal deer to shoot at, that is all I will say about it. I personally would not but I can only worry about the way I hunt.
I think there needs to be some more long range shooting, a check on the rifles zero and possibly a look at the rifle set up for the shots.
I am first to admit I miss from time to time. Every time I have missed, I have been able to track back to the exact reason why I missed and it has yet to be the rifle or ammo...... That leaves only me or the set up which is generally the case.