The real reason to go to a 50 or 55mm objective is its light gathering ability. If you hunt in the dark woods, or in poor light they can be an advantage. they can be a disadvantage in other ways like you described. I prefer the 50+ mm objective as long as it is in a 30 mm tube. The 30 mm tube allows for bigger glass inside and more MOA adjustments in the Turrets. also the 30 mm tube scopes can be shorter in length than high powered 40 mm scopes.
On many of my personal rifles I still have 40 and 44 mm Objective lens for many of the reasons you mentioned, But for the low light conditions I prefer the 50 + objective lens foe many reasons. I also like them because of there ability to gather more light in the lower power range for running shots.
As I get older, the better the light the better the shot, so I am slowly changing to the larger objectives.
J E CUSTOM