I think a couple things need to be added by the OP in order to help with the ideal caliber.
1) what exactly are you hunting ie, what is medium game, large game for YOU ?
2) are these animals in your back yard, or on the rare expensive hunt ?
I believe most would agree these questions play Into the equation. If your able to hunt the game you seek on a regular bases, you are able to be picky on shot opportunities and pass less than ideal shots, because you can try again tomorrow. If it's a hunt of a lifetime, you need the horsepower to take whatever shot opportunity presents itself.
To the other question, size of an animal doesnt always give the true will to live of a critter. A 280 or 06 would be plenty fine if you live among an elk herd , but if you've saved for 5 yrs to go on an elk hunt, a 338wm or 300rum would allow lethal shots you might pass with a standard caliber. Accuracy is important with either cartridge group , but a Texas heart shot on a 380" bull running away at 500yds is a higher % shot with a 215 or 300gr bullet at magnum velocities than a 180gr traveling 250fps slower at that same range.
I once knew of a guy that went 6 for 6 on mature bull elk with a 22 250. 5 of 6 fell in their tracks with HEART shots. Yhe other ran a short distance. He lived where the elk did, so he had the luxury of waiting for a perfect broadside /slightly quartering away shot the little bullet required to make it happen.