I like the suggestions of WiscGunner.
I would add weight - set it up as a handicap system - >=10lb rifle is a 0.8 multiplier, >=9lb is a 1.0, >=8lb is a 1.1 multiplier, >=7lb is a 1.3 multiplier, >=6lb is a 1.5 multiplier, <6lb is a 1.7 multiplier.
Hits and misses - what is the worst thing you can do when shooting at an animal? Hit is, but not kill it. So.... 3 points for a killing shot, -6 points for a wounding shot, and -4 points for a clean miss (and no more shots on that target).
IMHO only one shot at a critter - except varmits.
Time limits - absolutely. Grazing animals don't wait.
Bonus points for snap targets, but that could be hard to pull together.
Rather than muzzle energy, look at energy on target. There is no way someone can cleanly kill a Bison (american buffalo) at 600 with a 140gn bullet launched at 3000fps. Power commensurate with the quarry.