Five in the rifle, three in my left shirt pocket, and ten in the backpack.
I take at least 20. Most often though I have a full case (50) of reloads.
I took a hard fall once and my rifle took a harder fall while hunting. I didn't knock the scope out but I did need to verify zero and tracking. If you had something similar happen it could take a fair few rounds to troubleshoot.
You can never have too many (as long as your truck is the one hauling them). If I'm backpacking it is ten in my pouch and three in the mag.
There are only two times you can have too much ammo;
When you're drowning, and when you're on fire!