Annealing should be done when seating pressure increases, this is the brass fighting the change to the larger diameter. Annealing REDUCES neck pressure, it does NOT INCREASE it.
The reason to anneal is to give CONSISTENT neck tension, the frequency of this is dependent on the brass alloy itself and how much movement of the brass is taking place when sizing, this where work hardening occurs, when the brass is squeezed back to be close to SAAMI specs.
Even if you use a bushing neck die, if your chamber neck is .008" over loaded neck diameter, then .010" of 'working' is occurring everytime you size the neck.
I have a rule of thumb, hunting rifles should have 3-5 firings before annealing, varmint/target rifles 3 firings and comp rifles every firing.
All annealing should be done before sizing, I don't care if the spent primers are removed first or not. I used to tumble first, but, moved away from that, as my brass rarely gets dirty enough to obscure the results.
Hope this helps you understand WHEN to anneal.
Cheers.
gun)