The increase in group size with an increasing number of rounds is normal. Five shots is actually a very small sample size and if you truly want to have a very firm idea of where your rifle is going to place the next bullet, it takes a 20-30 round group to do it. FWIW, MOA 10-shot groups from a lightweight hunting rifle is very good.
Many years ago, I used 3-shot groups because conventional wisdom stated "I'll never take more than three shots at an animal." But, I noticed that when shooting another group with the same load, the point of impact might be slightly different, or maybe the group would open up, having two shots touching, and one a little out. Why? So, I started shooting multiple groups at the same POA and saw that the composite group showed me a much truer picture of what that rifle/load combo was truly capable of.
Providing the whole system is sound, a rifle will fire all of its bullets in a cone. With the 6PPC benchrest rigs, this cone will be very small, with others, bigger, but still a cone. The more rounds you fire, the bigger this cone will get until the sample size is big enough to represent a high level of confidence in statistical variation. What we, as hunters want, is to put enough rounds in a group to have a reasonable idea of where the next round is going. We're this not so, we wouldn't shoot groups at all, but just a single shot. In order to preserve barrel life, I do this by firing three, or maybe five, let the barrel cool, then fire three or five more at the same POA until I have a composite group of the sample size I want. The more shots in the group, the higher your confidence level can be. Most of the time now, I use a 10-shot composite. While not absolute, it provides a much higher level of confidence than the typical 3 or 5-shot group. 20 shots is better.
For those that say 3 is enough, I challenge you to do as I did years ago, and stack a couple of more groups on top of the first one, letting the barrel cool between groups, and see how big the composite is. It will absolutely be bigger than the single group and it is very likely that your "1/2 MOA all day" rifle really isn't.
A larger sample size doesn't just apply to group size either. The same principle applies to zeroing and load development. If you shoot enough rounds, you will discover that your .3gr increase in powder charge doesn't make nearly the difference you thought it did. I wanted to find max for a particular rifle once, so loaded 10 rounds, increasing by .4gr per round, figuring the first time I saw excessive pressure signs, I'd back off .5gr and call that max. The composite 10 round group with 10 different charges went into one ragged hole, well under MOA. I could have picked any one of those powder charges and expected similar results.
Actually, group size is no longer my preferred indicator of how well my rifle is shooting. Extreme spread only takes into account the data provided by two rounds, the two that make the widest spread. Average distance from center is better and you really don't have to measure it, you can see it. If during load development, I have a charge where the extreme spread of two different charges is similar but one of them has 8 of the ten wadded into a tiny little hole and the other is randomly scattered, I'm going to pick the first.
John