I would take it off and the whole deal, but that's just me. hell, i'd do it if my temps didn't seem too high. they could always be lower with some TLC (as3

). here's my method...
1) take off the hsf and clean both surfaces as thoroughly possible with isopropyl alcohol. i mean NO residue left. nvidia logo on gpu should be nice and clear and the bottom of the heatsink/fan (hsf) should be one uniform color and hopefully shiny.
2) next, get some saran wrap type stuff or a sandwich bag (clean, no fingerprints) and cover your index finger so the pad of the tip of your finger is nice and smooth. place a small dap of your thermal compound on the gpu and smear it around with your wrapped finger. rub it around until the whole surface of the gpu is somewhat uniformly covered, altough it may be a bit transparent. the purpose of this is to work the compound into the tiny cracks and dips on the surface of the gpu. then do the same with the contact area on the hsf.
3) next, i place a larger (bb-sized?) drop on the very center of the gpu. then i bring the hsf over the gpu and align it with the mounting holes as best as possible before i bring the hsf down
flat and
even to make contact with the gpu. continue to pess down evenly with a slow and slight left/right twisting motion to help the dab of compund flatten out and cover the whole mating surface.
4) then fasten the screws/clips/whatever and sit back and realize your card looks exactly the same!

but know in your heart you did it a big favor.
if you have a themal probe of some kind, you could--before the job--measure the temp on the back of the card where the back of the gpu would be after a few loops of 3dmark. then do the same after to see what kind of difference you made. but if your card already has a diode, that should be good enough.
troubleshooting--
if your temps read the same or higher, definitely count on taking it back off and checking the surfaces to see where the compound didn't reach. then think about if it was from an insufficient amount of compound or an uneven squeezing hsf application. learn from your mistake and try again.
i definitely recommend as3 for this kind of job (opposed to as5) as you will use a lot with the large contact area and a possible rework or two. and using that much as5 will get expensive.