With self timing brakes (those that can be removed and returned to the same aligned position, or has the provision to adjust to the same position after removal), I prefer to clean the rifle with them removed. Over time, they can accumulate carbon/copper deposits that may not be scrubbed in areas which cannot be reached by the cleaning rod patch/brush. Excess build up can effect performance, and even make contact with the bullet. This is more common with my heavy use competition rifles. While a brake can be sufficiently cleaned while attached, it's easy to let the removed brake soak in some solvent and give it a good scrubbing. It's also a good opportunity to clean the muzzle crown and the threads where extended use may cause carbon seizing. IMO.