It sounds like you have the right thought process in place, namely putting a new Swift tube on the Swift and building, say a 22 Creedmoor.
The only thing I would say is if rebarreling your Swift, go with a minimum 8 twist. Do not go any slower. I have rebarreled a pair of push feed M70's, one with a SS 5R Rock barrel and the other a SS Brux. Both are 1-8 twist. If I were to do it over, I would have gone 1-7.5 or perhaps 1-7.
I also have a 1-14 Swift. The lightest bullets I have shot in all three is 50 grain. All three shoot groups about the same size. The heaviest I have shot are 80 grain in the fast twists, again both of them shooting the same size groups. The 1-14 will not, of course, shoot an 80 grain accurately which brings up the point of going with a fast twist to be able to shoot the heavies as well as the lightweights.