So Melodyne wasn't exactly getting it done, so I "did my homework" and figured out my old method from Sonar days. First you need to come up with a "average temp" then set your metronome to match. Then move your source audio to roughly line up with the temp.
Select the source track for the metronome, then open the Audio Snap palette (ALT+A), then click edit clip map. This will set audio transients. Use the Threshold slider to get it in the sweet spot, you are shooting to get a transient at just about every beat. It won't always be perfect, but you don't need it to be perfect. then you simply hit the TAB key to move to the next Transient.
Turn on the metronome for playback, and playback the track, when the audio starts to misalign with the metronome, find the closest transient to a down beat and click the set measure/beat at now (shift +m). this will allow you to "stretch the tempo" to fit back to the source audio. Keep playing the song from this spot, and when it starts to get off again, do the same thing.
This might be a bit more tedious than dropping the track in the time bar and letting Melodyne do it's thing, but I find the manual method produces less noticeable change in the tempo.
Thanks again for all your help.