There is an old theory of fluid dynamics that was developed in the early part of the 20th century called "Boundary Layer Theory". Basically, a thin film of fluid (air in this case) stays stuck to a surface of an object as it moves through it. However, at some point in the surface the boundary layer will separate from the surface. To calculate the drag you would have to integrate the differential equation that describes it across the exposed surface of the object where the boundary layer has separated. What I am proposing is that the the boundary layer formation is affected by the density of the fluid (aka altitude above sea level affect the density of air) which in turn leads to a higher drag which slows down the bullet allowing gravity more time to act on it in the downward vector. Obviously this is just a hypothesis that has not been tested but it assumes that a lower fluid density causes the boundary layer to separate sooner.
en.wikipedia.org
en.wikipedia.org
This link from NASA is probably the simplest explanation I can find to describe the behavior of an object moving through a fluid.
"To make things more confusing, the boundary layer may lift off or "separate" from the body and create an effective shape much different from the physical shape. "