For a lot of folks, the biggest thing they'll shoot is a whitetail. That makes thing easier because a 6mm or 6.5 can cover that and still tear up a target at long range. If you hunt bigger animals, something in 30 cal makes sense. Exotics are interesting but there's some standards that, for a do all gun, make some sense. I like the '06. It's the one just before you start getting into the magnums. Recoil is gentle enough that you don't need a brake. It's fun for punching paper and good in the field. It has a twist rate that lets you shoot a heavy bullet if you need to but loves mid-weight bullets too. It falls right into the range with the most abundant selection of great components/ammo that are likely to be available. Oh, and it's so common, I'd be willing to bet you or your friends haven't picked on up in 20 years. I took my hunting '06 out today along with a 6.5x47 longer range rig. I hadn't even taken the duct tape off the end of the barrel of the '06 from hunting season. I took a coin out of my pocket, unscrewed the cap and dialed the elevation on the old VX-2, 3x9 and started banging steel at 518 yards. Then I dialed elevation on the 6.5x47 and did the same. It was fun getting nearly the same hits with both guns although the '47 was easier to shoot. The '06 was half the weight though so if I had to choose, I'd choose that.
There's a reason you got a pickup in your driveway instead of a Ferrari. The whole "one gun" discussion begs, not to be amazed but rather, to be satisfied and still avoid intermittent aggravation that a higher maintenance rig puts you through.
There's probably a standard caliber out there that's just meat and potatoes that'll make you happy as a clam.