You can't do that without swapping other pieces. However, if you can find two centers or two corners that look the same, then you can do it. (For the same centers, you would solve it with pure center commutators; for the same corners, you could do a T perm to rotate the center by 90 degrees and swap the two corners, and then fix the edges with commutators.)