You are wise to avoid fillers. I tried them on the 45/70 and the 45/110 using smokeless powder and found them erratic in both velocity and SDs.
I did lots of research on fillers and the consensus was that the filler had to much air space in it and this could change the burn rate. Even though it could be compressed, being fibrous It was contributing more oxygen to the powder effectively causing the powder to act more like a squib load.
I even went as far as weighting the filler, and still found that the same weight of filler material took up different amounts of space and measuring compression force found differences.
A long time commercial re loader advised me not to use fibrous fillers and to switch to compressed and treated (Normally wax) wads. This worked, but it was difficult to find wads the right thickness so I abandoned the whole process and found a powder that could fill the case and achieve the velocity and pressure i wanted.
I know that there are some people using fillers to fill cases, But my experiences with them was not to my liking. I never had any thing bad happen but the results were Questionable so I elected to be on the safe side.
Mike Venturino has written lots of information about the use of fillers and warnings. I will try to fine some of them and post them.
My only comment is to be careful if you do use fillers.
J E CUSTOM