It would limit a lot of shots not just those at distance. Just look at the below picture, these bison were at 100 yards, it is a chip shot with any kind of optic and normally an easy shot with irons but due to the fading light I couldn't see my front sight well enough to comfortably take the shot with black sights on a dark brown bison. Wasn't a big deal then as I always had the next day and worst case I could use my scoped Win Mag instead of the Sharps.
However if that was the last day of my hunt, I was holding a tag I waited years for and had spent several thousand dollars to get to that moment then I'm going to aim to the best of my ability, pull the trigger and hope that my shot hits its mark as there is no mulligan.
View attachment 406951
That's because your sights are probably general purpose with a giant fiber optic front sight and a two dot rear, fine for general close range stuff in lower light but certainly not designed for longer range accuracy. Anybody who intends to shoot at range with irons is not going to use those types of sights and would probably go with a globe front with various inserts as well as an adjustable rear tang.